It’s not clear why our new Masters of the Universe (in Tom Wolfe‘s words) are taking so long to learn a basic principle: machines do not think. As Associated Press’s Australia correspondent Rod McGuirk reports,

Deloitte Australia will partially refund the 440,000 Australian dollars ($290,000) paid by the Australian government for a report that was littered with apparent AI-generated errors, including a fabricated quote from a federal court judgment and references to nonexistent academic research papers.

The financial services firm’s report to the Department of Employment and Workplace Relations was originally published on the department’s website in July. A revised version was published Friday after Chris Rudge, a Sydney University researcher of health and welfare law, said he alerted the media that the report was “full of fabricated references.” “Deloitte to partially refund Australian government for report with apparent AI-generated errors,” October 7, 2025

McGuirk identifies the problem as hallucination, the tendency for AI chatbots (technically, large language models) to just make stuff up.

It can get pretty florid. As economics prof Gary Smith noted last year, for a while ChatGPT was claiming that the Soviets had sent bears into space.

The reason this happens is that the bot is designed in such a way that it will make up an answer rather than report no information. Now, here’s the difficulty: Obvious nonsense can be spotted easily and corrected by personnel at say, ChatGPT or Claude.

For example, in the Deloitte misfortune, if the federal court judgement was quoted as saying that every employee should eat spiders for lunch, it wouldn’t take long for the “spiders” to hit the fan. Everyone but the bot would know that that cannot be what a court ruled.

But much nonsense may not be obvious at all. A quotation from a court judgment that supports a policy perspective that sound reasonable may be fabricated. And no one who doesn’t check, as that health law researcher did, will ever know. After a while, the amount of completely erroneous information will start to become a more significant part of the overall information base.

Progress in reducing these errors has been, as computer engineering prof Robert J. Marks reports, slow and halting because of the underlying problems. His advice: “I suspect there will never be enough Band-Aids to stop all of the bleeding. So instead of ‘Trust but verify,’ I recommend ‘Verify, then trust.’”