Apple Says AI Reasoning Is an Illusion. What Now?
Apple's machine learning team just published a paper that should make every growth leader pause before handing a hard decision to a reasoning model. In The Illusion of Thinking, researchers Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio and Mehrdad Farajtabar tested the frontier reasoning models and found that their accuracy does not just degrade under pressure. It collapses to zero once a problem crosses a certain complexity threshold.
That is the headline. The models everyone is quoting as the next step toward general intelligence, OpenAI's o1 and o3, DeepSeek-R1, Claude 3.7 Sonnet Thinking and Gemini Thinking, all failed the same way once tasks got genuinely hard. And here is the detail that unsettled me most: as the problems got harder, the models started thinking less, not more, even though they had plenty of tokens left to use.
What the paper actually tested
Most AI benchmarks lean on maths and coding problems, and score models purely on whether the final answer is right. The Apple team argues this is a flawed way to judge reasoning. Two reasons. First, those benchmarks suffer from data contamination, meaning the model may have seen the answers during training. Second, a right answer tells you nothing about whether the model actually reasoned its way there.
So they built controllable puzzle environments instead. These let the researchers dial complexity up or down while keeping the underlying logic identical. The puzzles avoid contamination, rely only on rules the model is given, and can be checked step by step with a simulator. That last point matters. It let them inspect not just the answer, but the full internal reasoning trace, the so-called "thinking" the model does before it commits.
This is a far more honest test than a leaderboard score. It looks at the working, not just the result.
The three regimes every marketer should memorise
The most useful finding, in my opinion, is that the paper identifies three distinct performance regimes when you compare reasoning models (LRMs) against standard LLMs given the same compute budget.
| Task complexity | What wins | Why it matters |
|---|---|---|
| Low | Standard LLMs are more accurate and more token-efficient | Paying for a reasoning model on simple tasks wastes money and can hurt accuracy |
| Medium | Reasoning models pull ahead, but burn far more tokens | The extra "thinking" earns its keep only in this middle band |
| High | Both collapse completely, accuracy hits zero | No model type can be trusted on the hardest problems |
Table: The three reasoning regimes identified in Apple's The Illusion of Thinking (2025). Source: ml-site.cdn-apple.com.
Read that top row again. On simple tasks, the cheaper non-thinking models actually beat the expensive reasoning models. That flips a lot of received wisdom on its head. More thinking is not universally better. It is only better inside a narrow window.
The counterintuitive collapse
Here is the part I keep coming back to. As problems got harder, the reasoning models ramped up their effort, measured in inference-time tokens, up to a point. Then, right before the collapse, they started reducing that effort. They gave up. And they did this despite operating well below their generation length limits, so it was not a token ceiling. The paper calls this a fundamental inference-time scaling limitation.
The reasoning traces tell an even richer story. On simple problems, models often found the correct answer early, then kept exploring wrong alternatives anyway. That is the "overthinking" phenomenon. At medium complexity, correct answers only appeared after long detours through wrong paths. Beyond the threshold, they never found the answer at all, and frequently fixated on an early wrong idea and stuck with it.
The authors also found these models fail at exact computation. They do not reliably follow explicit algorithms, even when given them, and they reason inconsistently across different puzzle types. That last finding echoes a debate I have written about before, on how these systems actually behave once they meet the real world versus how the demos suggest they will.
Photo: Unsplash. Reasoning models generate long chains of thought, but Apple's research shows that effort peaks then declines as problems get harder.
Why this matters for AI in marketing
Marketing teams are quietly wiring reasoning models into work that carries real consequence: budget allocation, attribution modelling, multi-step campaign planning, competitor analysis, forecasting. The instinct is that a "thinking" model handles complexity better. This paper says the opposite can be true.
From my observation, the danger is not that the model gets a maths sum wrong. It is that it produces a confident, well-written, completely wrong answer to a genuinely complex planning task, and there is no error message to warn you. The reasoning trace looks plausible. The output reads well. And it is nonsense. That is a hard failure to catch at scale.
This also reframes cost. If standard models beat reasoning models on simple tasks, you may be paying a premium for worse results. I think a lot of teams have defaulted to the most expensive model out of caution, when the evidence points the other way for routine work.
What to do about it
Here is the practical playbook I would run on the back of this research.
- Match the model to the complexity band. Use cheaper standard models for simple, well-defined tasks. Reserve reasoning models for the medium band where they genuinely add value.
- Never trust AI on genuinely complex, multi-step decisions without a human check. The collapse is silent. Build the review step in.
- Give the model the algorithm and verify it uses it. The paper shows models often ignore explicit instructions. Do not assume compliance.
- Test on your own hard cases, not benchmarks. Contamination inflates leaderboard scores. Build a small set of your real, thorny problems and score models on those.
- Watch for overthinking on easy tasks. Longer output is not better output. It can be the model wandering past the correct answer.
The wider lesson connects to a point I have made about vetting AI vendors on evidence rather than marketing claims. "Reasoning" is a product label, not a proven capability. Apple's team has handed us a rigorous way to test the label against reality, and the reality is more limited than the pitch.
None of this means reasoning models are useless. They clearly help in the middle band. But the collapse is real, the effort declines when you most need it, and the failures are invisible. Treat these tools as capable assistants inside their range, and put a human on the hard problems. That is the honest position the evidence supports.
Tags