Mind Matters Natural and Artificial Intelligence News and Analysis
jellybean-candy-in-a-jar-stockpack-adobe-stock.jpg
Jellybean Candy in a Jar
Jellybean Candy in a Jar

The Wisdom of Crowds: Are Crowds Really Wiser Than Individuals?

According to the theory, with a large number of guessers, the median number is very likely to be close to the true value

Statistician Sir Francis Galton went to a country fair in 1907 where a prize was to be awarded to the person who made the most accurate guess of the butchered weight of an ox that was on display. Galton collected and analyzed the 787 guesses and, not surprisingly, found that some guesses were far too high and others were much too low. However, the average guess (1,197 pounds) was only 1 pound lower than the actual weight (1,198 pounds). The average was more accurate than the guesses of the vast majority of both the amateurs and the experts.

In the 1980s, a finance professor named Jack Treynor (1930–2016) performed a similar, and now legendary, experiment with jelly beans. Professor Treynor showed 56 students a jar full of jelly beans and asked them to write down how many beans they thought the jar contained. The actual number was 850 and the average guess was 871, an error of only two percent. Only one student did better than the average guess.

These experiments have been cited over and over as evidence of the wisdom of crowds. There is even a book with the title, The Wisdom of Crowds: Why the Many Are Smarter Than the Few (2004). National Public Radio spread the word in 2015 when Planet Money redid Galton’s experiment by sending staffers to the Burlington County Farm Fair in New Jersey and asking people to guess the weight of a cow named Penelope. NPR also posted Penelope’s picture online and collected more guesses. The average guess was within 5 percent of Penelope’s actual weight. Planet Money concluded that, “This is the idea that underlies the stock market, that a bunch of random people buying and selling shares collectively somehow get the right answer.”

This evidence is seductive and there is also a mathematical proof of the wisdom of crowds: The Theorem: If we collect independent, unbiased estimates of the value of something, then, as the number of estimates increases, it is increasingly certain that both the average estimate and the median estimate will be very close to the true value.

Applying The Theorem to the stock market, it can be argued that, because the market price is determined by a balance between buyers and sellers, the market price reflects the opinion of the median investor. The Theorem says that, with a large number of investors, the median assessment is very likely to be close to the true value.

Applying The Theorem to elections, it can be argued that the outcome is determined by the median voter. If there is a large number of voters, the median voter’s assessment of a candidate’s ability will likely be very close to the candidate’s true ability.

Applying The Theorem to forecasts of future events (including the outcome of sporting events), it can be argued that the market prices set by gamblers are determined by the median bettor and are likely to be close to the true probabilities. If the Minnesota Vikings football team is a 7-point favorite to beat Dallas, then evidently Minnesota is, as likely as not, to win by 7 points.

In prediction markets, bettors buy and sell contracts that pay off if a specified event happens. For example, on the last trading day before the 2020 US Presidential election, bettors paid 89 cents for an Iowa Electronic Markets contract that would pay $1 if Joe Biden won the popular vote implying that bettors thought Biden had a 89 percent chance of winning. Again, The Theorem implies that 89 percent was likely to be a very accurate estimate of Biden’s actual chances of winning the popular vote.

Unfortunately, The Theorem makes two crucial assumptions that are often unjustified: that the crowd’s estimates are unbiased and that the assumptions are made independently. Treynor was well aware of these assumptions, even if those who tell the jelly bean story are not. After the initial student guesses were recorded, Treynor advised students that they should allow for air space at the top of the jelly bean jar and that the plastic jar’s exterior was thinner than that of a glass jar. The average estimate increased to from 871 to 979.2, an error of fifteen percent. The many were no longer smarter than the few. Treynor wrote that, “Although the cautions weren’t intended to be misleading, they seem to have caused some shared error to creep into the estimates.”

In addition to shared error, the wisdom of crowds fails when the crowd is wrong or uninformed. If most people think that Dallas is the capital of Texas, the average answer won’t miraculously become more accurate as the number of people surveyed increases. The same is true if people are asked to identify the largest land animal that ever lived on earth and they make essentially random guesses.

There is a lot of shared error and uninformed noise that can undermine the wisdom of crowds in stocks, elections, and wagers. In the stock market, investor opinions are often biased and seldom independent. Stock prices are buffeted by fads, fancies, greed, and gloom—what Keynes called “animal spirits.”

During the Tulip Bulb Bubble in the 1600s, the prices of exotic bulbs topped $75,000 (in today’s dollars). In the South Sea Bubble in 1720 (see the painting by Hogarth), stock was offered in a company that had been formed “for carrying on an undertaking of great advantage, but nobody is to know what it is.” The shares for this mysterious offering were priced at £100; each, with a promised annual return of £100;. After selling all of the stock in less than five hours, the promoter left England and never returned.

More recently, during the dot-com bubble, Yahoo’s price-earnings ratio was a mind-boggling 2,375. In order to justify its $125 billion market value, Yahoo would need to be as profitable as Wal-Mart in 2000, twice as profitable in 2001, three times as profitable in 2002, and so on forever. Yahoo was deliriously overvalued until it fell off the proverbial cliff, with Yahoo’s stock price plummeting 90 percent.

In elections, voters around the world have clearly made some catastrophic assessments of candidates. Some areas have even elected pets and dead humans to office, though perhaps the citizens were correct in their judgment that these frivolous choices would be more effective than the live humans on the ballot.

In sports wagers, my own research with Marcus Lee and Andrew Capron has found that gamblers overreact to recent performances and consequently overestimate the probability that successful teams will continue to succeed and that unsuccessful teams will continue to fall short. It has consequently been profitable to bet on underperforming teams and against overperforming teams.

Prediction markets flop when the participants are uninformed or seduced by a herd mentality. For example, a widespread consensus based on incorrect information led prediction markets to badly miss the nomination of John Roberts to the Supreme Court, the 2016 US presidential election, and the UK Brexit referendum.

The crowd was not wise when How Green Was My Valley was selected Best Picture over Citizen Kane in 1941, when the 18th Amendment was adopted (prohibiting the manufacture, transportation, and sale of alcohol), when the most popular vegetable US children eat is french fries, when most Americans believe in ESP, when companies could double their stock price during the dot-com bubble by putting a dot-com in their name, when people paid $500 for a Beanie Baby, or when people paid $20,000 for Bitcoins that have absolutely no intrinsic value.

The crowd is often foolish and should never be trusted blindly. When making investment decisions, it is usually more profitable to be a contrarian investor and do the opposite of what the crowd is doing. Buy when the crowd is selling and sell when the crowd is buying. As Warren Buffett wrote, “Be fearful when others are greedy. Be greedy when others are fearful.”

When making business decisions, it often pays to remember Steve Jobsdismissal of the crowd: “Some people say, ‘Give the customers what they want.’ But that’s not my approach. Our job is to figure out what they’re going to want before they do. I think Henry Ford once said, ‘If I’d asked customers what they wanted, they would have told me, A faster horse!’”

The most amazing thing about the human brain is arguably not our collective wisdom, but our individual imagination and creativity. We create astounding literature, music, and art. Our scientific understanding of the world and our technological achievements are breathtaking. Our inventions are so numerous and life-changing that a list of the most important ones is a fun exercise and a needed reminder of how remarkable humans can be: the printing press, light bulb, telephone, television, computers, anesthesia, steam engine, vaccines, steel, wheels, nails, compasses, optical lenses, paper, internal combustion engines, penicillin, semiconductors, the internet,

Just look around you and think about everything you have done and will do today and how different your life is from that of other animals and from our distant ancestors. Think, too, about how different real human intelligence is from so-called artificial intelligence.

Let’s give thanks for our astonishing brains and our remarkable creativity. While acknowledging the many ways in which we are similar, let’s give thanks for all the ways in which we are unique individuals.


You may also enjoy these articles by Gary Smith:

Why intelligent women marry less intelligent men. Are they trying to avoid competition at home as well as at work? Or is there a statistical reason we are overlooking?

and

The paradox of luck and skill: Why did Shane Lowry win the British Open golf championship? Because someone had to. In any competition including academic tests, athletic events, and company management where there is an element of luck that causes performances to be an imperfect measure of ability, there is an important difference between competitions among people with high ability and competitions among people of lesser ability.

Also: The US 2016 Election: Why Big data failed Economics professor Gary Smith sheds light on the surprise result.


Gary N. Smith

Senior Fellow, Walter Bradley Center for Natural and Artificial Intelligence
Gary N. Smith is the Fletcher Jones Professor of Economics at Pomona College. His research on financial markets statistical reasoning, and artificial intelligence, often involves stock market anomalies, statistical fallacies, and the misuse of data have been widely cited. He is the author of The AI Delusion (Oxford, 2018) and co-author (with Jay Cordes) of The Phantom Pattern (Oxford, 2020) and The 9 Pitfalls of Data Science (Oxford 2019). Pitfalls won the Association of American Publishers 2020 Prose Award for “Popular Science & Popular Mathematics”.

The Wisdom of Crowds: Are Crowds Really Wiser Than Individuals?