Don't be fooled—LLMs don't reason

Thore Graepel, MIT Technology Review, 2 Oct 2026 Source: https://www.technologyreview.com/2026/10/02/1145639/dont-be-fooled-llms-dont-reason/

Summary

• AlphaGo's move 37 was reasoning, not intuition. Graepel, a core member of the AlphaGo team, says the move wasn't a flash of machine intuition. AlphaGo's policy network rated it a roughly 1-in-10,000 play for an expert human. Its search machinery chose it, by building a game tree and weighing future consequences.
• AlphaGo had a System 1 / System 2 split. Its neural networks supplied the hunches and its tree search supplied the deliberation. Neither half would have found the move alone.
• LLMs are essentially System 1. They predict the next token repeatedly. Chain-of-thought helps, especially in maths and coding. But it is the same next-token process run for longer, not a separate reasoning mechanism.
• Three shortcomings keep chatbots from reasoning in a scientist's sense:
  1. No explicit, persistent, inspectable "epistemic state": no ledger of hypotheses, confidence, evidence and open questions.
  2. No separation between what the system knows and how it manipulates that knowledge. Both are tangled in the weights.
  3. Chains of thought are often concocted after the fact, so the model reaches an answer one way and reports another.
• Why it matters: in medicine, engineering and science you need to know how a conclusion was reached, so errors can be traced to bad reasoning, bad evidence or bad assumptions.
• His proposal: he left Google DeepMind to pursue it. A system keeps an explicit epistemic state (settled, doubted, ruled out, open). Reasoning becomes moves that update that state. An independent component scores each move by how much uncertainty it resolves, and beliefs update only when evidence backs the change. LLMs would be components that suggest approaches and use tools, not the whole system.

Quote

"I do not think we reach trustworthy machine intelligence by making system 1 bigger."

Only one short quote is included; everything else is paraphrased.

Note for the "AI is dumb" angle

His claim is narrower than "dumb". He says LLMs are very good at pattern completion but lack an auditable reasoning mechanism, and that scaling alone won't fix that. The piece is also a pitch for his own architecture, so the "genuine reasoning" framing is his position rather than settled fact.