I’m back in the thick of my love-hate relationship with AI. Tonight, I’m offering a sneak peek into a project that tackles a pressing issue: the hallucination problem in language models. Why do these systems sometimes go off the rails? The burning question is: how do we cut down the error rate?

Let’s dig in. It starts with the right critique, and the right ideas.

Summary of the problem

One of the major challenges with large language models (LLMs) today is their tendency to “hallucinate”—that is, to generate responses that sound plausible but are factually off the mark. Despite all the advancements we’re seeing in AI, we’re nowhere close to fixing this issue. and there are deep-seated reasons why.

Here’s what’s really going on when LLMs hallucinate and what it would take to move from pattern-based language generation to genuine understanding.

Why hallucinations happen: No “truth model,” only probability

LMs hallucinate because they lack a model for “truth.” They don’t check facts in the same way that we think about truth; they work probabilistically—as many people now understand. When an LLM responds, it’s not “thinking” in terms of what’s true or false. It’s silly to expect it to somehow “find” truth or falsehood in the data.

Image Credit: FantasyLand86 - Adobe Stock

What is often overlooked is how the underlying technology can drive follow-on research toward genuine improvements and solutions. From what I can see, major companies are merely scratching the surface—primarily relying on post-process filters and reinforcement learning. These approaches are reactive, patchwork solutions rather than substantial investments in comprehensive research that addresses the hallucination problem at its core.

Some background on how LLMs work

A large language model (LLM) calculates the probability of each word w given the context C, represented as P(w | C), and then selects the most likely next word. This setup is effective for generating fluent language but inherently lacks fact-checking. When we encounter hallucinations, it’s because the model fills in gaps with what sounds plausible rather than what is verifiable.

Without a built-in “truth model”—something that cross-checks, validates, or confirms accuracy—the system is effectively a great “guesser.” It’ll often sound authoritative but lack the foundational layer that ensures that what it’s saying is factually correct. This is built into the architecture. The model doesn’t even know what a “word” is. It computes tokens (there are many non-word tokens in, for instance, GPT-4o, which is one reason it can occasionally impress when asked to be creative).

From data to thoughts: The real challenge ahead

Here’s the question: How do we get from data to actual thoughts? Pattern recognition alone won’t get us there. I think the research community is starting to recognize and acknowledge this limit. Real comprehension needs more than just predicting the next likely word; it needs mechanisms that can reason, validate, and question.

To get there, we’d need an architecture that doesn’t just generate language but can think critically about it. For example:

1. Build and reference a knowledge model: Real understanding needs an adaptable internal model of the world—a “world model”—that AI can use to cross-reference its outputs against a structured, verified knowledge base. We have many examples of world-models to date, but none that work with a deep layered neural network.
2. Develop reasoning capabilities: Human thought isn’t just about storing and retrieving data; it’s about reasoning through relationships, understanding context, and assessing credibility. This is where current LLMs fall short. To bridge that gap, AI would need to include some explicit or reliably “implicit” persistent model of basic concepts like space and time. Guessing the next word won’t cut it.
3. Implement validation and real-time fact-checking: Moving from pattern generation to thought also means enabling models to validate responses against real-world sources in real-time. This would let AI shift from producing fluent responses to accurate, reliable information.

In short, LLMs are impressive tools for generating language, but without a truth model, they can’t get close to actual thought. Hallucinations are a symptom of that gap—a sign that we’re still dealing with fluency, not true comprehension. The future of real AI will need models that don’t just predict but can know and verify. Here are some proposals:

Steps toward lower error rates and reducing hallucinations

Now, let’s look at a few research directions that might help reduce hallucinations, though none of these will fully solve the issue:

1. Hybrid architectures for logic and mathematical accuracy
   Hybrid models are trying to merge neural networks with symbolic AI or rule-based logic systems. This works well for highly structured tasks, like math and logic, where clear-cut answers are required.

For example, hybrid systems might use Boolean satisfiability (SAT) solvers to validate binary logic or constraint satisfaction problems (CSPs) for relational rule tasks.

In standard LLMs, each word w is selected based on P(w | C), the probability of the next word given context C. Hybrid architectures add a verification layer:

P(w | C) → P(w | C) ⋅ Verify(w)

where Verify adds a logical check, filtering for accuracy in contexts governed by logic or strict rules.

2. While useful for math, this approach doesn’t do much in open-ended contexts. From personal experience I can tell you that logic is maybe 1% of the cognitive performance we want. Dedicating a lot to it is going in the wrong direction. Still, it can play a role.

A note on hybrid models

Hybrid models aren’t magic—they don’t “augment” in any deep sense. What they actually do is act as a filter. They’re useful, sure, filtering out errors in structured tasks like math or logic. But make no mistake: a hybrid model isn’t a ride to AGI. It’s a targeted tool, not a leap toward genuine intelligence.

3. Self-consistency and verification mechanisms
   Self-consistency techniques involve generating multiple responses, then selecting those that converge.

If we have n responses, represented as:
S = { r₁, r₂, …, rₙ }

a consistency function C(S) checks for overlap among these responses. The final answer is then selected as the mode of the responses:
Final Answer = mode(S)

One (basic) method for minimizing on the outlier function. We need more research here.

Image Credit: witsarut - Adobe Stock

The remaining problem

The problem is that the mode is still probability, so you can see how the field is stuck.

4. Retrieval-augmented generation (RAG)
   Retrieval-Augmented Generation (RAG) lets the model pull real-time data from external databases.

Given a query Q, it retrieves information D and generates a response R:

R = Generate(Q | D)

RAG grounds responses in real data, I think this is helpful in fields with constantly updating information, like finance or to some degree medicine. RAG reduces error rates but doesn’t fully eliminate hallucinations due to source data limitations.

5. Liquid neural networks for continuous adaptation
   Liquid Neural Networks (Liquid NNs) continuously adjust weights in response to new inputs:

dW/dt = f(W, I)

where W is the weight matrix, I represents input, and f dictates adaptability.

This will get us to Liquid NNs refining outputs based on real-time data, though their impact on hallucination reduction will, I suspect, still be limited.

LNNs have a lot of promise, but so far the tech has issues scaling to the “emergent” properties we get from the largest models. I want to take this math and see how we can “boost” it with a larger seed in the pretraining. Everyone is looking at this, but there’s a way to do it.[]

6. Improving training data and fact-verification protocols
   Expanding and refining the training data helps reduce hallucinations by establishing more “truthful” biases. Fact-verification protocols can also reinforce this effect by assigning higher probabilities to responses aligned with known truths.
7. Human feedback and reinforcement learning (RLHF)
   Reinforcement Learning from Human Feedback (RLHF) aligns model responses with human input through a reward function:

R(w | C) = +1 if correct, -1 if incorrect

RLHF minimizes inaccuracies through human feedback but its effectiveness depends on the quality and scope of that feedback. Here’s the problem, the RLHF doesn’t touch the model which has the weights for generation. It, again, is essentially a filter on a core tech. We need updates in the core tech. Hybrid is great, but it’s a post-process filter. We need a fundamental idea.

8. Self-Supervised Learning and World Models
   Yann LeCun advocates for models with built-in “world models” that allow for cause-and-effect learning. In LeCun’s framework, the internal state M updates over time based on new inputs:

M_{t+1} = f(M_t, I_t)

where M_t is the model state at time t and I_t is the new input. While promising, these world models need significant advances to reach their potential.

Will it work? This is how innovations generally works: a thousand flowers are planted, and something blooms. Jury is out.

9. Conclusion: The next step in AI

So, here’s my best guess on what we need to move forward: models like LLMs have already solved part of the mystery of cognition. They generate fluent, cogent language—even perfectly structured grammar—without having explicit rules of grammar embedded in their systems. Instead, they’re using a highly optimized linear algebra function that predicts and produces linguistically sound responses based on probability alone. It’s impressive, but it’s only one part of the cognitive equation.

The real challenge now is moving from optimized language generation to something closer to genuine understanding. Researchers like LeCun are already exploring architectures that go beyond probability. That work looks to self-supervised systems with “world models” that mimic human knowledge-building processes. This is smart. This kind of world model could allow AI to handle not only data but structured ideas and cause‒effect relationships. But we don’t have anything like that yet.

If we’re going to solve the full cognition puzzle, my hunch is that we’ll need an architectural approach that can handle propositions—thoughts, judgments, reasoning structures—as first-order objects. To think in the way humans do, AI needs mechanisms that treat propositions as entities to be evaluated, related, and verified against a broader framework of knowledge. This kind of architecture would shift us from simple response generation toward actual thought, moving us closer to AI that doesn’t just mimic language but genuinely understands it.