We don’t usually see an article in traditional media that offers serious pushback to overstated AI claims. But the Wall Street Journal’s technology columnist Christopher Mims recently provided a good one:

Before you get nervous about all the times you were rude to Alexa, know this: A growing cohort of researchers who build, study and use modern AI aren’t buying all that talk.

The title of a fresh paper from Apple says it all: “The Illusion of Thinking.” In it, a half-dozen top researchers probed reasoning models—large language models that “think” about problems longer, across many steps—from the leading AI labs, including OpenAI, DeepSeek and Anthropic. They found little evidence that these are capable of reasoning anywhere close to the level their makers claim …

Apple’s researchers found “fundamental limitations” in the models. When taking on tasks beyond a certain level of complexity, these AIs suffered “complete accuracy collapse.” Similarly, engineers at Salesforce AI Research concluded that their results “underscore a significant gap between current LLM capabilities and real-world enterprise demands.”

Importantly, the problems these state-of-the-art AIs couldn’t handle are logic puzzles that even a precocious child could solve, with a little instruction. What’s more, when you give these AIs that same kind of instruction, they can’t follow it.

“Why Superintelligent AI Isn’t Taking Over Anytime Soon,” June 13, 2025

And much more.

Our fine writers, technology consultant Jeffrey Funk and Gary Smith have been covering this gap between hype and performance for years. See, for example, “No, large reasoning models do not reason” by Gary Smith and “AI’s contradictory impact on productivity: squeezing a balloon.”

If only evidence-based thinking sold as well as hype.