Erik J. Larson, who has written for us over the years, has a fun piece up at Nautilus: Is the advanced chatbot (Large Language Model or LLM) that OpenAI, is bringing out next year — ChatGPT5 — as smart as a five-year-old?

The machine, he says, “will reportedly be equipped to solve complex logical and multi-step problems like a human with a Ph.D. What it likely won’t do, however, is know much of anything about how the world works.” Thus, Larson, author of The Myth of Artificial Intelligence (Harvard 2021), sets its skills against those of the five-year-old in a number of areas, including these two:

COMMON SENSE AND CONTEXTUAL UNDERSTANDING

The LLM:

LLMs can draw logical connections based on patterns in training text, such as understanding that if “it rains, the ground gets wet.” However, they might not grasp that bringing an umbrella is a practical next step unless it’s explicitly stated in the text.

They’re also capable of generating responses that are contextually accurate, as long as those contexts are well-defined in their data. LLMs may struggle with ambiguous phrases like “Can you pass the salt?” but typically know when to say “thank you” after a compliment.

The 5-Year-Old:

Five-year-olds instinctively know that you shouldn’t stack books on top of a cake, even if they might try it just to see what happens. Can say “thank you” after a compliment but might also stick their tongue out.

They understand the concept of gravity when their ice cream falls but might still argue that it was “gravity’s fault” when they trip over their own feet.

Can understand that bedtime means pajamas and brushing teeth, but might insist that wearing a superhero cape to bed is a necessary part of the routine, just in case they need to fly in their dreams.

Ask them about space, and you’ll hear about aliens with ice cream planets.

IMAGINATION AND CREATIVITY

The LLM:

Can whip up a tale of dragons and knights or a futuristic sci-fi epic by pulling together elements from vast databases of stories.

Their creativity is apparent in their ability to produce text that sounds imaginative, if not always original.

The 5-Year-Old:

The 5-year-old mind is a limitless playground. They invent invisible friends who speak in rhymes, and worlds where every animal wears a hat and glasses.

A child’s artwork might include a rainbow next to a giraffe because why shouldn’t the giraffe have a lovely day?

Erik J. Larson, “Who’s Smarter: AI or a 5-Year-Old?,” Nautilus, August 15, 2024

You can probably guess who Larson declares to be the winner but check your guess here. He notes that LLMs are “bounded by the data they’ve been trained on, and they lack genuine understanding, common sense, and emotional depth.”

Larson writes a Substack column, Colligo, as well.

You may also wish to read: How Erik Larson hit on a method for deciding who is influential. The author of The Myth of Artificial Intelligence decided to apply an algorithm to Wikipedia — but it had to be very specific. Many measures of influence depend on rough measures like numbers of hits on pages. Larson realized that influence is subtler than that.