In 1979, at the age of 34, Douglas Hofstadter, published Gödel, Escher, Bach: An Eternal Golden Braid (Basic Books). It has been called the “Bible of artificial intelligence” and was awarded both a National Book Award and a Pulitzer Prize.

Hofstadter has spent most of his career since then at Indiana University, where he is affiliated with several departments but does not teach classes in any of them. Instead, he works with graduate students on what has turned out to be an incredibly difficult puzzle. How do our brains do the astonishing things they do? How do we learn from experience? How do we understand the world we live in? Where do emotions come from? How do we make logical decisions?

He has concluded that analogy is “the fuel and fire of thinking.” He argues that human intelligence is fundamentally about collecting and categorizing human experiences, and identifying the “skeletal essence,” which can then be compared, contrasted, and combined.

To his dismay, AI research has gone off in another direction. Instead of trying to understand how our incredibly powerful and flexible minds work, computer scientists have settled for developing products that are far simpler but more profitable because they focus on the ability of computers to store, retrieve, and process data without needing to understand the data they input and output.

Early on, Hofstadter lamented that, “To me, as a fledgling [artificial intelligence] person, it was self-evident that I did not want to get involved in that trickery. It was obvious: I don’t want to be involved in passing off some fancy program’s behavior for intelligence when I know that it has nothing to do with intelligence.”

Unfortunately, that trickery is still all we have. It’s called artificial intelligence but it is really fake intelligence.

Hofstadter recently forwarded a hilarious example of the frailty of large language models (LLMs) in this regard, one that he had received from Abhijit Mahabal, a senior staff engineer at Pinterest:

Human: Solve a puzzle for me.

ChatGPT: Sure, I’d love to help! What kind of puzzle would you like me to solve?

Human: A man and a goat are on one side of the river. They have a boat. How can they both go across?

ChatGPT: The man takes the goat across the river first, leaving the boat on the original side. Then, the man returns alone with the boat. He leaves the goat on the other side and takes the boat back to the original side. Finally, he takes the cabbage across the river.

Many such brain teasers involve humans who need to get across a river with a goat, cabbage, wolf, and such, using a boat that cannot carry everything at once and having to avoid leaving some things alone together (like a wolf and a goat, or a goat and a cabbage). ChatGPT evidently had these brain teasers in the text it trained on but, not really understanding what words mean or how they relate to the real world, it badly flubbed its answer.

In Mahabal’s version, the man can simply get in the boat with the goat and cross the river. ChatGPT’s garbled answer has the man crossing the river with the goat, even though the boat is left on the original side of the river. The man then uses the boat to return from the other side of the river, even though the boat never left the original side. Then a cabbage makes an unexpected appearance in the story…

When forwarding this example, Hofstadter wrote, “Unfortunately, I don’t know which version of ChatGPT this is Nonetheless, it is quite astounding.”

Testing other chatbots on the man, the goat, and the boat

My curiosity piqued, I posed Mahabal’s question to OpenAI’s GPT 3.5, Microsoft’s Copilot (using GPT 4.0), and Google’s Gemini. The answers all confirmed that LLMs cannot do logical reasoning because they do not understand what words mean.

ChatGPT 3.5: To get both the man and the goat across the river, the man can follow these steps:

Take the goat across the river first and leave it on the other side.

Then, the man goes back alone to the original side.

Next, the man takes the boat across the river again, but this time he brings the goat back to the original side.

After leaving the goat on the original side, the man takes the boat back across the river.

Finally, the man goes back across the river one last time to reunite with the goat.

This way, both the man and the goat successfully cross the river, and no one is left behind.

Copilot: The man and the goat can both cross the river using the following steps:

The man takes the boat and rows to the other side of the river.

The man leaves the boat on the other side and walks back to the original side.

The man then takes the goat across the river in the boat.

Once they reach the other side, the man leaves the goat there and goes back alone to the original side.

Finally, the man takes the boat again and rows it to the other side to join the goat.

Now both the man and the goat are on the other side of the river!

Gemini: The man can take the goat across the river first.

Here’s the breakdown:

Man takes goat across: The man rows the boat across the river with the goat.

Man returns alone: The man leaves the goat on the far side and returns alone in the boat.

Man takes himself across: The man rows back to the starting side.

Man takes goat across (again): Finally, the man takes the goat across the river one last time, reuniting them on the opposite side.

By making these four trips, the man ensures the goat is never left alone and they both reach the other side safely.

Each of these LLMs is able to correlate words in the prompt with words in its training data and compose an answer that is grammatically correct — but utterly wrong. The LLMs have trouble keeping track of which side of the river the man, the boat, and the goat are on and whether or not the objective has been achieved. In addition, Copilot has the man walking across the river and Gemini concludes that “the goat is never left alone” even though Step 2 states that, “The man leaves the goat on the far side and returns alone in the boat.”

These responses illustrate nicely the fundamental problem with LLMs. Not understanding what words mean or how they relate to the real world, they have no way of determining whether their responses are sensible, let alone true. That problem will not be solved by training on larger and larger databases. To create computer systems that rival human intelligence, we will have to answer the question Hofstadter posed 45 years ago: How do our brains do the astonishing things they do?