Mind Matters Natural and Artificial Intelligence News and Analysis
transition-effect-in-bar-chart-statistics-and-bright-windows-stockpack-adobe-stock
Transition effect in bar chart statistics and bright windows
Transition effect in bar chart statistics and bright windows

Studies Based On Data Mining Can Turn Out To Be Nonsense

As business prof Gary Smith explains at Bloomberg, we can find a great deal of nonsense if all we rely on is a search engine

Many of the studies represented to us in media are not nearly as reliable as we would like to believe, as Gary Smith explains at Bloomberg. He starts with statistician John Ioannidis pointing out that of 34 “highly respected” medical studies and found that only 20 were confirmed by the Reproducibility Project.:

I wrote a satirical paper that was intended to demonstrate the folly of data mining. I looked at Donald Trump’s voluminous tweets and found statistically significant correlations between: Trump tweeting the word “president” and the S&P 500 index two days later; Trump tweeting the word “ever” and the temperature in Moscow four days later; Trump tweeting the word “more” and the price of tea in China four days later; and Trump tweeting the word “democrat” and some random numbers I had generated.

I concluded — tongue as firmly in cheek as I could hold it — that I had found “compelling evidence of the value of using data-mining algorithms to discover statistically persuasive, heretofore unknown correlations that can be used to make trustworthy predictions.”

Gary Smith, “Believe in Science? Bad Big-Data Studies May Shake Your Faith” at Bloomberg (April 26, 2022)

You may also wish to read: Why Big Data can be the enemy of new ideas Copernicus could tell us how that works: Masses of documentation entrench the old ideas. Erik Larson, author of The Myth of Artificial Intelligence (2021) notes that, apart from hype, there is not much new coming out of AI any more.


Mind Matters News

Breaking and noteworthy news from the exciting world of natural and artificial intelligence at MindMatters.ai.

Studies Based On Data Mining Can Turn Out To Be Nonsense