Findings patterns in data is easy. Finding meaningful patterns that have a logical basis and can be used to make accurate predictions is elusive. We can see this from 18th-century attempts to cure scurvy through 21st century claims about the stock market or history.
Read More ›
A psychological quirk called loss aversion makes us fear losses far more than we value gains. The panic investors experience from sudden losses can blind them to the overall, long-term picture. We can demonstrate this tendency with a simple thought experiment.
Read More ›
When a field is hot, there is a race to publish. With a large number of researchers trying a wide variety of treatments, it’s almost certain that that someone will find a coincidental pattern that is statistically significant but meaningless. The only solution, alas, is further testing.
Read More ›
Stanford University’s AI index offers us fanciful measures of the triumph of AI, rivaling the far-fetched metrics of dot-com commerce. The reality has been the opposite. For decades, U.S. productivity grew by about 3% a year. Then, after 1970, it slowed to 1.5% a year, then 1%, now about 0.5%. Perhaps we are spending too much time on our smartphones.
Read More ›
The paper that busted the myth of “hot hands” is justly famous. But statisticians are not prophets. Craig Hodges’ streak of 19 in a row in the 1991 contest is still too incredible to be explained by luck or cherry-picking. The numbers show that hot hands don’t happen every day, but they do happen.
Read More ›
The explosion of data has vastly increased the number of coincidental patterns that can be discovered by tenacious researchers. If there are a relatively fixed number of useful patterns and the number of coincidental patterns is growing exponentially, then the ratio of useful patterns to useless patterns must necessarily be getting closer to zero every day.
Read More ›
Bridge is one of the few games where computer algorithms have not yet demolished the best human players but, despite claims to the contrary, algorithms do a much better job of random shuffling of the deck.
Read More ›
Computer algorithms are much, much better than humans at discovering statistical patterns but much, much worse than humans at discerning whether the patterns are meaningful. Wise investors can use the resulting blips to their advantage.
Read More ›
My main interest in cryptocurrencies like bitcoin is in whether they are sound investments. I’m a value investor—I even wrote a book about that—and the short answer is no.
Read More ›
Computers, like a friend of mine, are great at very narrowly defined tasks, like playing Go, retrieving facts, and calculating correlations. But they are utterly unreliable for anything requiring true understanding, wisdom, or common sense.
Read More ›
One company sought to leverage Watson, the AI Jeopardy champ, as a stock picker. But lightning-fast search and response don’t have much to do with predicting whether a stock goes up or down…
Read More ›
Think about it. Line drives hit right at fielders, mis-hit balls dying in the infield. Fly balls barely caught and barely missed. Balls called strikes and strikes called balls. Even the best batters make twice as many outs as hits. Even the best teams lose more than a third of their games. This season, the Houston Astros had the highest win percentage (66.0%) in baseball, yet they lost two out of six games to Baltimore, which only won a third of their games—not because Baltimore was the better team, but because Baltimore was the luckier team in those two games. The Astros are one of the 10 best teams this season (along with the Yankees, Tampa Bay, Minnesota, Cleveland, Oakland, Read More ›
Professors at prestigious business schools and financial engineers with advanced degrees built mathematical models to assess the risks, but their models were deeply flawed.
Read More ›
What happens next? They all beat the market. Were their ticker names a factor? I decided to study that.
Read More ›
Many marketing decisions, medical diagnoses, and stock trades, loan and job applications, and election strategies are evaluated by computers. But, as my little experiment shows, the computer does not know whether a pattern is information or noise.
Read More ›
In any competition including academic tests, athletic events, and company management where there is an element of luck that causes performances to be an imperfect measure of ability, there is an important difference between competitions among people with high ability and competitions among people of lesser ability.
Read More ›
Unfortunately, patterns are not always a source of information. Often, they are a meaningless coincidence like the 7-11 babies this summer.
Read More ›