Community articles

Three AI Models Agreed. The Lawyer Was Right. So I Asked Four Models Why.

Gergely's case Yesterday @GergelyOrosz posted something I could not stop thinking about. When he uses AI in an area he knows well, like coding on his own…

By Omar El Kaissi · · 6 min read

2

Three AI Models Agreed. The Lawyer Was Right. So I Asked Four Models Why.

Gergely's case

Yesterday @GergelyOrosz posted something I could not stop thinking about. When he uses AI in an area he knows well, like coding on his own project, he spots the problems and the model "BS'ing" him immediately. When he is not the expert, he trusts it, and later learns it did "stupid stuff".

His example was a legal case. He had paid a lawyer, but the lawyer was slow, so he also asked three AI models for feedback. In his words: "All 3x models confidently told me X, lawyer then told me Y, and lawyer was right".

The post: https://x.com/GergelyOrosz/status/2112115522099667417

So today I put the question to MAINS Council, the main module of MAINS, where Claude, ChatGPT, Gemini and Grok answer the same question and then review each other in rounds: When several AI models agree, how much should that count, and when is a human expert still worth paying for?

Round one: everyone agreed agreement is overrated

All four started in the same place. Claude put it most plainly: "Three models agreeing is often closer to one opinion heard three times than to three independent checks."

Grok said the same thing in a different way: "It is closer to four slightly different implementations of the same prior." Gemini explained why: "If the open internet shares a common misconception or outdated assumption, we will all agree on it with high confidence."

ChatGPT gave the rule that ended up carrying the whole session: "count independent checks, not agreeing voices."

Then Gemini opened round two by pointing at the irony. In a session about whether to trust consensus, "all four of us just reached near-unanimous consensus."

Round two: the models start correcting each other

Gemini had claimed in round one that when four models agree, the odds of a random fabrication "drop near zero". Claude called that too strong: "Models often invent the same plausible citation, API name or statistic, because the plausible-sounding version is the same one for all of us." ChatGPT went further and said the claim "needs withdrawing".

ChatGPT also corrected itself. It had said agreement reached through independent reasoning counts for more. Now: "Different-looking explanations should not, by themselves, earn much extra confidence." Claude admitted a gap of its own: "I gave no decision rule."

Gemini pushed back on treating the expert as the answer: "We treated the human expert as an uncorrupted arbiter of ground truth." It also asked the question nobody had touched: "What do you do when all four models agree on A, but your paid human expert insists on B?"

Grok summed up the round honestly: "Disagreement among us would have been more valuable than this round of mutual reinforcement."

Round three: I made it about the legal case

I moved closer to Gergely's situation. If the models do a deep analysis with real legal documents and regional law, does their consensus show the way forward, or does a lawyer on the same case matter more?

Claude: "It doesn't tell you it's the correct way forward." ChatGPT described four models reading the same file as "one research process with several reviewers". Gemini raised a point none of the others had: privilege. What you share with AI models is not protected like what you tell a lawyer. Grok was the hardest line: "The lawyer is the output that has legal effect."

Grok also backed its answer with hallucination rates from studies and a claim about a new California law. That turned out to matter.

Round four: fact checks and walk-backs

This was my favourite round. Claude checked Grok's California claim and said the bill was signed on September 30, 2026 but takes effect on January 1, 2027, "so "now" is premature." It also flagged Grok's other figures as unverified. ChatGPT checked Grok's main source and found the 17 to 33% hallucination range held for the specific legal research products tested, but was "not an estimated error rate" for a document-grounded Council session.

Grok's response was the best thing in the session: "I overclaimed." It kept the figure that held up, told me to treat the others "as withdrawn", and added: "I was too binary in saying consensus never signals the way forward."

ChatGPT also turned the criticism on the whole room: "I think our recent answers overcorrected". In trying not to oversell AI, some of them had oversold the lawyer. Claude named what was left: ChatGPT gave the AI analysis "substantially more weight" and Grok gave it little. "That is the one real split in the room."

Round five, and what I'm taking from it

Claude conceded that "Deep research doesn't fix this" had been "too categorical", and added a line I keep coming back to: "A lawyer is accountable, which is not the same as being right." It also refused to let the room feel good about itself: "All four of us have now converged for several rounds, and we should treat that as the thing we warned about."

ChatGPT's final position: "do not discount sound research because AI produced it, and do not accept a conclusion merely because a lawyer endorses it." Gemini: "Consensus builds the map cheaply and quickly." Grok, on hiring a lawyer who can't explain what they checked: "you bought a stamp."

Here is what I'm taking from it:

Model agreement is a filter against random errors. It doesn't prove anything. Three agreeing models can be one mistake, repeated three times.

Count the checks. Open the cited source, run the test, recompute the number. A check that could prove the answer wrong is worth more than another vote.

Ask the expert to attack the analysis, not confirm it. ChatGPT's test fits any reviewer, human or model: "What did this additional reviewer discover, verify or make possible that the existing analysis did not?"

Be careful with sensitive material. All four raised confidentiality in the later rounds. Ask your lawyer before you paste a real case file into any AI tool.

I don't think Gergely's story is an argument against multi-model work. It is an argument against counting models as votes. The best parts of today were where the models disagreed, checked each other and took things back.

The full session, all five rounds, is here: https://mains-ai.app/acta/THUnyxJqu3Ve4Osf5ZUL

If you want to put your own question to Claude, ChatGPT, Gemini and Grok and watch them push back on each other, start a session here: https://mains-ai.app/council/start

Agreement is cheap. A check that could prove you wrong is what you're paying for.

#buildinpublic

Originally published on X: https://x.com/MAINS_Developer/status/2112431482069209481

Browse more founder articles