There are known knowns, known unknowns, and unknown unknowns. It’s the unknowns unknowns that get you. – Donald Rumsfeld (1932–2021)

In this famous quote, Rumsfeld was outlining a major gap in everyone’s knowledge. Sometimes, we don’t know what we don’t know. While we can prepare for things that we know we don’t know, we can’t prepare for the unknown unknowns — except to expect the unexpected.

Unsurprisingly, the unknown unknowns also pose a problem for AI. However, AI is different from humans in that it cannot expect the unexpected. For an AI, everything must be defined in a way computers can understand, even the unexpected. But, the truly unexpected cannot be defined in a way computers understand.

The inability to define the truly unexpected is more than a throwaway observation about AI systems. It is actually a deep mathematical result captured by Kolmogorov complexity: describing an object in the least redundant way possible. The least redundant description of an object is the shortest computer program that can generate the object, and the length of the program is the object’s Kolmogorov complexity.

Kolmogorov complexity, at its core, is also a definition of the unexpected. The unexpected is that which is unpredictable. However, if something is redundant, then that means it is to some degree predictable. Consequently, being unpredictable, and thus unexpected, it must also be not redundant.

Because the Kolmogorov complexity of an object is its least redundant description, then the Kolmogorov complexity also describes the unexpectedness of an object. The more unpredictable and unexpected, the greater the Kolmogorov complexity.

Wait a minute Eric…

Previously you said the unexpected cannot be defined, yet here you just defined the unexpected in terms of Kolmogorov complexity? What gives?

Good point! There’s the rub. You might think that since we can define the unexpected mathematically with Kolmogorov complexity, then this translates to a definition computers can understand, since computers can understand math. However, computers cannot understand all math, as Gödel proved with his famous incompleteness theorems.

The same problem applies to Kolmogorov complexity. It is a mathematical definition that computers cannot understand. The proof is pretty simple. If we could measure Kolmogorov complexity with a computer program, then we could also write a computer program that generates an object with any amount of Kolmogorov complexity we want. However, this also means we can write a program that generates an object with greater Kolmogorov complexity than the program itself. Since Kolmogorov complexity is the length of the shortest program that generates the object, this is a contradiction. Therefore, computers cannot measure Kolmogorov complexity.

What this means is that AI will forever be unable to know whether an object is unexpected. So, unlike humans, AI cannot expect the unexpected. This is a fundamental ability humans have that AI does not have: the capability to deal with Rumsfield’s unknown unknowns.

An interesting corollary is that AI also cannot be truly creative. Creativity means creating something brand new. Yet, in order to create something brand new, a computer must also be able to knowingly increase Kolmogorov complexity, which is provably impossible.