DeepMind, a part of Alphabet (i.e., Google), has made many headlines in the past. The biggest was its development of AlphaGo, which used reinforcement learning to beat the number one Go player at the time (2017). DeepMind generalized this into AlphaZero, which is supposedly able to solve any two-player game of perfect information.

DeepMind has come back into headlines recently with the attempt to build an AI which can generate any algorithm. While they are starting with map data, the goal is to generalize this and generate any desired algorithm.

The search for such a “universal algorithm” has been essentially equivalent to the search for a perpetual motion machine in physics. The allure of both is obvious. In physics, if you can create a perpetual motion machine, you can remove the world’s need for energy. In software, if you can develop a “universal algorithm,” you can remove the world’s need for programmers. 

The problem with perpetual motion machines is that they violate the law of conservation of energy. You can’t generate more energy than you have. It is simply a limitation of physics. Interestingly, information theory has similar limiting laws, known as “conservation of information” laws. The most well-known incarnations of these are the “No Free Lunch” theorems by Wolpert and Macready, which states there is no universally good optimization algorithm — all rely on some form of expert knowledge.

Here at Mind Matters, we’ve talked about Turing’s Halting Problem as an example of this sort of limitation. Simplified, algorithms are combinations of axioms (fundamental assumptions) and theorems (combinations of axioms). Computers are essentially very fast theorem generators, but are fundamentally unable to generate axioms. Advances in machine learning generally come from the introduction of axioms into the mix. If you add an axiom, you may be able to generate thousands of new theorems, but no amount of theorem-generation will give you a new axiom.

This is, ultimately, the problem with universal algorithms. They are always limited by the axioms supplied to them. This is why machine learning always seems to be rounding a corner, but always eventually hits a wall. Every added axiom unlocks so many solutions, but then for the next set of solutions, new axioms are required.

As George Montanez pointed out in a 2017 paper, “The Famine of Forte,” few search problems favor any algorithm. Essentially, what Montanez was pointing out was that any search algorithm has a bounded set of problems that it can apply to. This means that universal algorithms are just not possible. 

So, while the search for the universal algorithm continues, don’t bet on anyone finding it.