Investors and the media just love headlines that cite a big impact of AI on productivity. For instance, the CEOs for Amazon and Microsoft have both said that a significant fraction of their software is written by AI, Duolingo’s CEO said it was going to be AI first, and the CEO’s for Fiver and CyberStrike have said that AI is coming for their jobs. But rarely if ever, do these CEOs provide detailed stories to back up those statements.

The Devil Lurks in the Details

And when details do emerge, they usually involve a retraction. IBM laid off 8,000 workers in 2023, and then hired them back in May 2025. In 2024, a top headline for Klarna was “Klarna’s AI Assistant Is Doing The Job Of 700 Workers” but a year later it was “Klarna Reverses AI-Only Customer Support Strategy.” Similarly, the headline for Starbucks in 2024 was “Starbucks’ AI Revolution: Deep Brew Stirring Up the Future of Coffee Retail” while in 2025 it was “Starbucks Is Hiring In-Store Human Workers After Replacing People With Machines — and Finding It Didn’t Work.” Even more concerning, MIT says it no longer stands behind a student’s AI paper that found AI enabled materials scientists to discover more materials.

The Wall Street Journal has been documenting the mixed impact of AI on jobs and productivity for years. Just in the last month, it described three largely superficial AI projects by Johnson & Johnson, which it characterized as the best 10% of its AI experimentation in a so-called “thousand flowers” approach. Similarly, paraphrasing IBM’s CEO, “the tech giant has used artificial intelligence, and specifically AI agents, to replace the work of a couple hundred human resources workers.” As a result, it has hired more programmers and salespeople.” In a third article, WSJ then summed up AI’s impact on business productivity in a headline: “Companies are Struggling to Drive a Return on AI,” in which many of the experts are recommending a task-based approach to AI (more on this later).

Image Credit: berkay08 - Adobe Stock

More detailed experiments also reach similarly nuanced conclusions, such as this one from AMD, one of the largest designers of chips, and of the software that links their hardware with operating systems. Its first conclusion is that “the kinds of tasks coding assistants are good at — namely, busting out lines of code — are actually a very small part of the software engineer’s job. A second is that “even for the coding copilots’ bread-and-butter task of writing code, we found that the assistants offered diminishing returns: They were very helpful for junior developers working on basic tasks, but not that helpful for more senior developers who worked on specialized tasks.” Third, “to use artificial intelligence in a truly transformative way, we concluded, we couldn’t limit ourselves to just copilots. We needed to think more holistically about the whole software-development life cycle and adapt whatever tools are most helpful at each stage.”

Similarly mixed conclusions were reached in a survey of Danish workers about the impact of generative AI on productivity. The survey of 25,000 workers and 7,000 workplaces revealed that “users reported average time savings of 2.8 percent” while the “AI chatbots created new job tasks for 8.4 percent of the workers, including some who did not use the tools themselves, offsetting potential time savings.”

Like Squeezing a Balloon

These stories remind me of what children learn when they first play with a balloon. When they squeeze it, some parts contract while other parts expand. This is one reason why it is so wrong for companies to focus on tasks, something that famous promoters of AI such as Erik Brynjolfsson and Ethan Mollick often do. Individual workers will then claim huge productivity improvements for their work, ignoring the impact of AI on the work of downstream employees within the same process. The biggest example involves the high-frequency of hallucinations that must be fixed by downstream workers, and those hallucinations aren’t going away: OpenAI admits that its new model still hallucinates more than a third of the time.

Such contradictions also exist at the industry or even societal level. For instance, in one study, the 16 most widely used large language models were used to generate 576,000 code samples, and it was found that 440,000 of the package dependencies they contained were hallucinated, meaning they were non-existent. A dependency is an essential code component that a separate piece of code requires to work properly. Dependencies save developers the hassle of rewriting code and are an essential part of the modern software supply chain.” That is not only important at the level of a single company, but even more so at the industry level, which is why the article was titled: “AI-generated code could be a disaster for the software supply chain.” Squeeze the balloon, and it expands someplace else.

On the surface, AI is giving workers, particularly support workers, a big revolution in productivity. But it is somewhat like a child squeezing a balloon; the air pushes out someplace else. Support workers have always been measured by the extent to which they support the core goals of the organization, be it shipping cars or computers, mining lithium, harvesting wheat, catching fish, insuring homes or cars, or treating patients. But AI makes it easier for support workers to create text and images and thus think they have become super productive. The problem is that that text and those images often serve goals that are different from the company’s goals. Companies must ensure that the increased productivity of support workers does not come at the expense of the organization.

Education is the biggest balloon

One final story must be mentioned because of a recent New York Magazine article that received much attention. Many students purportedly believe that AI is making them more productive at university, but most of us have known for a long time that this isn’t true. The article demonstrated that many students can’t explain what they wrote in a term paper using AI. So, the students may think they are more productive because they can finish their term papers very quickly. But if they can’t explain what they wrote, what have they learned? Very little. And a lot of resources will be devoted to fixing this problem. Keep squeezing that balloon.