Remember the “space bears” that Gary Smith wrote about here at MMN? — where chatbots claimed falsely that Russia had sent bears into space, complete with bogus references? That’s often called a “hallucination” on the part of the program.

University of Rochester astrophysicist Adam Frank says we should quit using terms like “hallucination” to describe what is happening. Human thought, he argues, is not at all like information processing in machines:

Query the web for a definition of hallucination and you’ll get something like this: “A hallucination is a false perception of objects or events involving your senses: sight, sound, smell, touch, and taste. Hallucinations seem real, but they’re not.” The important point here is the link to perception. To hallucinate means you must first perceive; you must already be embodied in the world. To use a term from phenomenology, you must already be embedded in a rich and seamless “lifeworld” that carries multiple contexts about how that world operates. But what’s happening within a large language model (LLM) chatbot is not even close to what’s happening in you.

Adam Frank, “Stop saying that ChatGPT ‘hallucinates,’”Big Think, February 23, 2024

So, in a nutshell, what’s the difference?

An LLM is not hallucinating when it gives you a made-up reference in the paper on Roman history you just asked it to write. Its mistake is not a matter of making a false statement about the world because it doesn’t know anything about the world. There is no one in there to know anything about anything. Just as important: The LLM does not, in any way, inhabit a world in which there are mistakes to be made. It is simply spitting out symbol strings based on a blind statistical search through a vast hyperdimensional space of correlations. The best that can be said is we make the mistake about the world when we use these machines we’ve created to answer questions.

Adam Frank, “Stop saying that

Next month, Frank, along with physicist Dartmouth College theoretical physicist Marcelo Gleiser, and philosopher Evan Thompson will publish a book with MIT Press: The Blind Spot: Why Science Cannot Ignore Human Experience So what do the authors think is the difference between a human and a chatbot?

To be human, to be alive, is to be embedded in experience. Experience is the precondition, the prerequisite, for everything else. It is the ground that allows all ideas, conceptions, and theories to even be possible. Just as important is that experience is irreducible. It is what’s given; the concrete. Experience is not simply “being an observer” — that comes way downstream when you have already abstracted away the embodied, lived quality of being embedded in a lifeworld.

Adam Frank, “Stop saying that

In their view, the “blind spot” is the unquestioned assumption that the chatbots rampage through data is equivalent to human experience. Their worry is not that AI will take over but that we will become “less human as we are forced to conform to its limitations.” That’s of particular concern to the authors because they argue that it is specifically human experience that makes science possible. Unless, of course, we are willing to allow a chatbot to inform us that the Russians sent bears into space …

The book will be available March 5.

You may also wish to read: Why some think emergence is replacing materialism in science. Materialism, in the form of reductionism, posits a world without novelty — but that is not the world we live in, says Adam Frank. Philosophers and scientists who champion emergence over reductionism argue that the whole is more than the sum of its parts.