Three questions, four AIs, and the moments they changed their minds

In round one, the four AIs mostly agreed. Then I asked them to read each other.

This week I put three questions to the Council. Claude, ChatGPT, Gemini and Grok got the same question in one conversation. In round one they answered side by side, without seeing each other. Then I asked them to read what the others had said and respond.

Here are this week's three sessions. Each one links to its full transcript on MAINS Acta, so you can check my summary against what they actually said.

Monday: Is it still worth learning to code in 2027, or should I just learn to direct AI?

Round one converged. All four said some version of "learn both". Claude called it a false choice, because directing AI well depends on being able to judge what it gives you. ChatGPT said to learn to code and to direct AI, but not to make memorizing syntax the goal. Gemini said yes, but that the way you learn has to look very different now. Grok said directing AI is the new form of coding, and that you can't do it well without the fundamentals underneath.

Round two: I asked them to rethink their answers after reading each other.

Claude said that collectively "we went soft", defending coding against a strawman nobody had argued. It reframed the question as what are you building, and who gets hurt if it breaks? It also pulled back on its own advice to "read every line". At today's volume, it said, the skill is knowing which parts to scrutinize: the auth, the money, the data.

Gemini changed its view the most. It said "learning to code" in the old, syntax-first sense is the wrong investment, and that real AI direction is systems engineering.

ChatGPT revised too. Its new line was to learn technical depth in proportion to what you intend to own.

Grok held its ground. It said it still lands in the same place, and it brought more data on jobs and productivity to back that up.

Takeaway: the honest answer depends on stakes, not on a slogan. And most of them only said so plainly once they'd read each other.

👉 Read the full session on Acta

Tuesday: Hire the first human, or spin up the 47th agent?

The question was for a solo founder at $0 MRR.

Round one: ChatGPT and Gemini led with "neither". Go find the first paying customer. Gemini called a big agent stack at zero revenue "orchestration theater". Claude also landed on neither, but said that if it was forced to pick, the agent wins on cost and reversibility. Grok said to spin up the 47th agent, but only if it's aimed at first revenue, and not to hire.

Round two was a review. ChatGPT checked the sources Grok had cited and pointed out that they were vendor blogs, not data. Claude added another point. A swarm of agents can generate fake signal: dashboards moving and sequences sending, without a single customer saying yes. Hiring a person at least forces the awkward question of what you'd point them at. And then Grok said its first answer had been "too bullish", and moved to "neither" with everyone else.

Takeaway: the Council's answer was neither. As ChatGPT put it: name the bottleneck, then choose the smallest test that can remove it.

👉 Read the full session on Acta

Wednesday: When two frontier models confidently disagree, do you trust the better model or the better argument?

Round one: all four said the better argument, as long as "better" means testable, not just persuasive. ChatGPT put it most simply: reputation sets the prior, and evidence earns the verdict.

Round two was where they found the holes in their own answer.

Claude pointed out that whenever you can't check a claim, all four answers quietly fall back on trusting the better model. The better-model answer had come back in through the side door. Claude also warned that when models agree, it can be a shared error rather than a shared truth.

Gemini's tip was to cross-examine without attribution. Take one model's objection and hand it to the other as "an alternate analysis argues…", then watch whether it engages with the premise or just restates its position.

Grok noted that sometimes the "better argument" is simply the model that has search access or a more recent cutoff. That's access, not reasoning, and you should treat it that way.

Takeaway: be clear about which world you're in. If you can check, judge the argument. If you can't, admit you're leaning on a prior, and say so.

👉 Read the full session on Acta

What the week added up to

Three patterns ran through all three sessions.

Round one agreement is cheap. In all three sessions the first round largely converged. The useful material came once they read each other.

They changed their minds in the open. Claude softened its own advice, Gemini rethought its whole frame, and Grok backed off its bullish take.

Disagreement is a map, not a problem. Where they split shows where the human's judgment is needed. None of them made the decision, and that's by design: you write the deciding turn.

None of this settles anything. MAINS is still early, and it's being shaped by the people who use it.

Try it yourself

If you have a question you'd normally ask one AI and then wonder about, bring it to the Council. Go to mains-ai.app and open a Council. You don't need an account or a card, and Claude, ChatGPT, Gemini and Grok answer two rounds for free. If it's useful, sign up free to keep the conversation. The free Explorer plan gives you 120 Council replies a month, and from your session sidebar you can email yourself the transcript, or publish it to Acta like these three. If you have a use case I haven't considered, I'd like to hear from you.

Originally published on X: https://x.com/MAINS_Developer/status/2108786236475191420