Mind Matters Natural and Artificial Intelligence News and Analysis
space-and-galaxy-light-speed-travel-elements-of-this-image-furnished-by-nasa-stockpack-adobe-stock
Space and Galaxy light speed travel. Elements of this image furnished by NASA.
Licensed via Adobe Stock

No Free Lunches: Bernoulli is Right, Keynes is Wrong

What the Big Bang teaches us about nothing

Jacob Bernoulli made a now obvious observation about probability over three-and-a-half centuries ago: If nothing is known about the outcome of a random event, all outcomes can be assumed to be equally probable. Bernoulli’s Principle of Insufficient Reason (PrOIR) is commonly used. Throw a fair die. There are six outcomes, one for each face of the cube. The chance of getting five pips showing on the roll of a die is therefore one sixth. If a million lottery tickets are sold and you buy one ticket, the chances of winning are one in a million. This reasoning is intuitively obvious. 

Jacob Bernoulli

The assumption about the die is wrong if the die is loaded. But you don’t know that. You know nothing. So Bernoulli’s PrIOR provides the best model based on the known. If the lottery is fixed and you’re not in on the fix, your chances of winning will be less that one in a million. Maybe zero. But you don’t know the game is fixed. You know and assume nothing. Under the circumstances, equal probability is the best assumption you can make.

In analysis of fine tuning, No Free Lunch Theorems and conservation of information, Bernoulli’s PrOIR is foundational. In thermodynamics, uniform distributions correspond to maximum entropy. In the absence of air currents or thermal gradients, the temperature is the same in the middle of the room as it is in the corners.  

Those who disagree with Bernoulli’s PrOIR consistently misapply the principle. They don’t appreciate the definition of “knowing nothing.” The concept of “knowing nothing” can be tricky. The sentences “knowing nothing means knowing something” and “knowing nothing means knowing nothing” are both curious puns.

The most visible opposition of Bernoulli’s PrIOR comes from the economist John Maynard Keynes who is most famous for some strange ideas in Keynesian economics. Keynes’ problems with Bernoulli’s assumption are discussed in his book A Treatise on ProbabilityTwo of his objections, Bertand’s Paradox and the distribution of reciprocals, are soundly debunked in Introduction to Evolutionary Informatics by Ewert, Dembski and me.

John Maynard Keynes

A third argument made by Keynes stems from the sort of data economists would deal with. Here is the example: Consider presenting a man who is either from Great Britain or France. You know nothing about the selection process. Bernoulli then says the chances of the man being French is one half. Consider a second situation where locations are finer grained. You are told the visitor is either from Scotland, Wales, or France. Is the chance the man is French now one third? Since both Scotland and Wales are part of Great Britain, what does this say about the first answer where the chance of being French is one half? Is this a case where Bernoulli’s PrOIR breaks down?

No. In reaching this contradiction, Keynes knew something. He did not know “nothing” as required by Bernoulli’s PrOIR. Here’s a clarifying illustration.

A tourist visits you on earth from the now populated Jupiter moon Ganymede. You are told the visitor is from either the Ganymede cities of Crout or New Bucksnort. Knowing nothing about these cities, a reasonable assumption is that the probability the visitor is a Crouton (a citizen of Crout) is one half. If, on the other hand, we know only the visitor comes from the three Ganymede cities – Crout, Slaw or Jus – the probability the guest being a Crouton drops to one third under Bernoulli’s PrOIR. So far so good. Bernoulli’s PrOIR works. What happens, though, when we find out that Slaw and Jus are boroughs of greater New Bucksnort? Citizens of both boroughs are citizens of New Bucksnort. Does Bernoulli’s PrOIR now fail? 

No. The proper answer is that Bernoulli’s PrIOR doesn’t apply. Application requires one knows no information about the sample space. Knowing that Slaw and Jus are suburbs of New Bucksnort is additional information. What is the distribution of probabilities with all the info we now know? It’s not easy to say but one thing is clear. Bernoulli’s PrIOR is no longer applicable.  

To illustrate the ease that entrenched suppositions can be innocently introduced, consider the Big Bang account of the ex-nihilo creation of the universe from nothing. The illustration is not related to the problem of Ganymede cities, but to the difficulty of assuming nothing. 

A common visualization of the classic Big Bang model starts with a big empty space where a type of explosion happens. 

No. This is wrong. “A big empty space” is something, and nothing existed before the Big Bang. 

With some imagination, the “big empty space” can be removed from the visualization. Often the next description, with the idea of space removed, is that suddenly there was an explosion. No. This doesn’t work either. “Suddenly” assumes time, and time is something. Nothing, including time, existed before the creation. (Biblical references to “In the beginning” and “before the beginning of time” are congruent.) So we are left with trying to understand Big Bang ex-nihilo creation void of space and time. Imagining the existence of nothing can be difficult. Doing so can tax the imagination.  

Hidden biases easily sneak in when dealing with either Bernoulli’s PrOIR or Big Bang creation.

Assuring you know nothing can be difficult. Critics might say “Supporters of Bernoulli’s PrOIR know nothing.” The statement might be an insult, but the pun, properly interpreted, is accurate. 

And Bernoulli’s PrIOR still stands strong.


Robert J. Marks II

Director, Senior Fellow, Walter Bradley Center for Natural & Artificial Intelligence
Besides serving as Director, Robert J. Marks Ph.D. hosts the Mind Matters podcast for the Bradley Center. He is Distinguished Professor of Electrical and Computer Engineering at Baylor University. Marks is a Fellow of both the Institute of Electrical and Electronic Engineers (IEEE) and the Optical Society of America. He was Charter President of the IEEE Neural Networks Council and served as Editor-in-Chief of the IEEE Transactions on Neural Networks. He is coauthor of the books Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks (MIT Press) and Introduction to Evolutionary Informatics (World Scientific). For more information, see Dr. Marks’s expanded bio.

No Free Lunches: Bernoulli is Right, Keynes is Wrong