A nerd was wandering through a tech market one day. Vendors were loudly hawking their wares: “Fresh LLMs straight off the GPU!” “Get your chatbot prompts here!” “Guaranteed 99% hallucination free!” “AGI in a box! Can’t tell it’s not sentient!” “No more flawed human relationships with FriendGPT!” “Join an eternal paradise in Singularity Cloud!”

Enticing as these products sounded, his attention was drawn to a little stall tucked away in the corner. Unlike the surrounding loud and flashy booths, it displayed a series of simplistic block pictures, surrounding the question, “Can your AI solve this?”

The other vendors jeered and made rude gestures at the vendor, a smallish middle-aged man: “What a sad little man, unwilling to let go of the illusion of reality…” “Doesn’t he know that minds are just meat computers?” “They should do something about people who block progress!” “That stall is positively unscientific!”

Our nerd ignored them because he wanted to see what all the hubbub was about. The little guy behind the table didn’t notice him at first, distracted as he was by all the antagonism. But surprisingly, he didn’t seem worried about it; he appeared to be enjoying it.

The nerd decided to strike up a conversation: So, what is the point of your sign and the pictures?”

“Glad you asked! Finally, someone who is not scared of the truth. Just look at all the pearl clutching. But it’s easy enough to explain if you have a few minutes.”

“Sure, but not much longer than that. I need to get back to work before an AI takes my job! I was just visiting the market to buy my own AI and maybe fend off the foodlines for a bit.”

“Baseless fears. Let me ask you a few questions to take your mind off them. First, have you heard of neurons before? Do you know their role in the brain?”

“Indeed I have!” Our nerd was relieved that it was a topic which he felt considerable confidence. “I learned in school that neurons are the basic processing unit in the brain. They pick up electrical signals from other neurons. When enough of a potential builds up in the neuron, it fires off its own signal to the ones connected to it.

“The brain consists of billions of neurons, all connected together by even more synapses. Just like the neural network in a Large Language Model! That’s why the LLMs are so close to human level intelligence — and we may already be there!”

The little man’s eyes narrowed: “Have you ever heard of any differences between the neurons in the brain and those in neural networks?”

That was another topic in which the nerd was reasonably well-read. “Not a lot of details, but there do seem to be some differences. The brain neurons are sort of chaotic; they don’t have the same kind of detail as a neural network. Brain neurons also fire much more slowly than neural networks. Frankly, it’s a bit surprising that neural networks haven’t caught up to the brain sooner. I suppose it’s because only now do we have the enormous amounts of computational power and data.

“And even then, training a state of the art LLM can require a city’s worth of power to do something my brain can perform from the calories in my sandwich. I admit it’s a bit odd, but a few more breakthroughs and we can reduce the LLM’s power needs. It should be even easier now that we have AI to assist us.”

Neurons vs. neural networks

Then the little man asked, “So, do you think the AI algorithms are becoming closer to the brain’s structure, or further away, as things improve?” Now that was something of which the nerd was a bit less sure.

“Hmmm, what an interesting question. Well, based on what I just said, it does sort of seem like the neural networks are becoming less like the brain in some ways, but more like the brain in other ways. For instance, deep learning uses a hierarchical structure like we find in the brain, although I’ve never heard of the brain using the famous backpropagation algorithm for learning. From what I understand, backpropagation is what broke neural networks out of merely being a curiosity, to today’s industrial behemoth.”

The little man continued to press: “How would you say the deep learning network structure is like the hierarchy found in the brain?”

“Come to think of it, those are apples and oranges. The brain hierarchy deals with quite different physical tasks, not really building up higher level concepts from lower level concepts as happens with deep learning. So, maybe the idea of hierarchy is more inspirational than literally the same as what we find in the brain. Actually, the structure of the two is also an example of neural networks moving away from what we find in the brain. I must confess this situation is looking even odder. Great questions!”

The little man ignored the flattery: “Where do we find these neural network innovations in human thought?”

“I was about to say we don’t, but then I caught myself! Actually, we do see things like backpropagation and concept-building hierarchies. And it is precisely within human thought, especially in school. We learn through constant feedback, and we build up advanced concepts from simpler concepts. This is so bizarre!”

“What is bizarre?”

The nerd was thinking. Hard. “First I realized that the modern building blocks of neural networks aren’t actually in the brain. I thought we were just making them up, since at the end of the day business cares about results, not scientific research. Without backpropagation and deep learning, AI would still be huddled up in its igloo as scientists tried to reverse engineer the brain to get useful algorithms.”

“However, thanks to your great questions, I have found the building blocks again. We get them from our everyday experience of our own thoughts. Yet this is what is so bizarre: The elements essential for the success of neural networks are not found in the brain, but found in our thoughts. How does something not in our brain end up in our thoughts? Yet that very thing is necessary for thoughts to work?”

Now the little man changed his tack: “That is a big puzzle. I’m afraid I don’t have a good answer for you. Now, I hope you don’t mind, let me ask you a question along another track. Do you consider neural networks now capable of human level thinking?”

“No, not really. At least nothing that I’ve seen. The most recent LLMs are truly impressive, generating fun art and providing conversational information troves. I spend a lot of time fiddling with it all, which I greatly enjoy. But, the there is not quite there. Everything has its quirks, and the chatbot can lose the plot. I know a lot of your colleagues in the market would say otherwise, but I would certainly not bet the farm on the current generation of AI. Give it a few more years, and clever engineers, and things will get better. I do hope to see AI like in the movies within my lifetime! I don’t know if I should be frightened, excited or both!”

The little man now showed his philosophical side. He replied, “I certainly agree! And that leads me to my final question. What do you make of it all?”

“Now that’s quite the question! What do I make of it all? When we first started talking, I thought I knew just about everything important about AI, including some of the things we talked about at first regarding the brain. But, as you kept asking questions, I’ve noticed there are some quite unexpected patterns. I noticed that as our AI improves, it is less like the brain, rather than more like the brain. Hadn’t thought of that before, which leads to the next point. On the other hand, as the AI improves, it becomes more like how our thoughts work. Yet, even so, AI has yet to catch up to our thoughts. A very curious pattern indeed.”

The nerd suddenly paused for a moment: “But, this is also making me think of a third, even more mysterious pattern to it all. When we pull these two points together, it is completely contrary to what I expected. I thought that since my thoughts come from my brain, then as our AI algorithms improve, they should become more and more like our brain. Especially as our AI algorithms begin to match our thinking abilities. But, it is the complete opposite. And so, I am left with the biggest conundrum of all. If my thoughts can only be copied by something quite unlike the brain, and copied fairly badly at that, then how is it possible for my thoughts themselves to be generated by the brain? What are my thoughts running on? What does that mean?”

The little man replied, “The same thought struck me, as well. And I still don’t have a good answer for you. But, these questions have certainly struck a chord with my colleagues at the market. Speaking of which, I smell some smoke and things are getting a little hot. I suggest we head out the back before our zealous friends discover the reality of fiery conflagrations in crowded markets!”

“Good thinking!”

The nerd and the philosopher rushed out the same exit.