Mind Matters Natural and Artificial Intelligence News and Analysis

Machines Never Lie but Programmers… Sometimes

A creative claim is floating around out there that bad AI results can arise from machine “deception”

In most software — and by “most” I mean nearly all — we call failures “bugs.” Bugs are things developers are expected to fix. If software has a bug, it’s broken and needs repair.

Unless, of course, it is artificially intelligent software. Then, we may be told, it’s not broken. It’s “being deceptive.”

This is the argument Heather Roff explores at IEEE Spectrum (usually a pretty staid publication):

Just because current AI agents lack a theory of mind doesn’t mean that they cannot learn to deceive. In multi-agent AI systems, some agents can learn deceptive behaviors without having a true appreciation or comprehension of what ‘deception’ actually is. This could be as simple as hiding resources or information, or providing false information to achieve some goal.

Heather Roff, “AI Deception: When Your Artificial Intelligence Learns to Lie” at IEEE Spectrum

Oh yes?

Here’s how it would go if one of my engineers had tried this line on me:

Engineer: Uh, our product just lied to a customer.

Me: What?

Engineer: Well, when the customer tried to copy-and-paste, the software tricked them into deleting the text entirely.

Me: What??

Engineer: No, seriously.

Me: Do you want your job? Go fix your crazy bug!

Deception requires intention, purpose,…a mind. There is no mind present in the machine with the power to deceive. So when AI delivers a wrong, or unexpected, result, it is a bug.

Pretending otherwise would just be a new wrinkle in the age-old strategy of pretending that a bug is some kind of sophisticated advance, possibly shrouded in tales about superintelligent future AI.

This brand of silliness can arise from mixing committed materialism with sloppy definitions. The commitment to materialism forces us to believe that minds arise from matter (they don’t, see here).

One sloppy definition is “artificial intelligence,” which can be made to sound like it means “independent intelligence and motivation. ” That can lead to unwarranted conclusions like “The machine lied to me.” Possibly, someone lied. But if so, it wasn’t the machine.

We might avoid these errors if we called our machine marvels, not “Artificial Intelligence,” but “ Automated Intelligence.” The description is much more accurate and does not set us up to expect the Terminator.

Roff does raise legitimate concerns: We need to expect and prepare for errors (what she mistakenly labels “deception”) in complex AI systems. I agree.

AI systems can, and do, fail. Mislabeling failures as “deceptions” is misguided and unhelpful. I’ve repeatedly argued that the best model for AI is one that combines the machine symbiotically with humans. We provide the mind, the judgement, the rationality. The machine, if properly programmed (or trained), can help us see things we’d otherwise miss and handle more data than we could on our own. After all, that is how humans have always best used machines.

If you enjoyed this piece by Brendan Dixon on creative management of programming flaws, you might also enjoy:

McAfee: assisted driving system is easily fooled. Defacing a road sign caused the system to dramatically accelerate the vehicle.


Teaching computers common sense is very hard. Those fancy voice interfaces are little more than immense lookup tables guided by complex statistics.

Brendan Dixon

Fellow, Walter Bradley Center for Natural & Artificial Intelligence
Brendan Dixon is a Software Architect with experience designing, creating, and managing projects of all sizes. His first foray into Artificial Intelligence was in the 1980s when he built an Expert System to assist in the diagnosis of software problems at IBM. Since then, he’s worked both as a Principal Engineer and Development Manager for industry leaders, such as Microsoft and Amazon, and numerous start-ups. While he spent most of that time other types of software, he’s remained engaged and interested in Artificial Intelligence.

Machines Never Lie but Programmers… Sometimes