Mind Matters Natural and Artificial Intelligence News and Analysis

Tagreplication crisis

Worried doctors and medical researchers on conference meeting, discussing possible solutions for resolving a world health crisis. Health and medical care concept. Selective focus.

Computers Excel at Finding Temporary Patterns

Which contributes to the replication crisis in science

The scientific method calls for the rigorous testing of plausible theories, ideally through randomized controlled trials. For example, a study of a COVID-19 vaccine might give the vaccine to 10,000 randomly selected people and a placebo to another 10,000, and compare the infection rates for the two groups. If the difference in the infection rates is too improbable to be explained by chance, then the difference is deemed statistically significant. How improbable? In the 1920s, the great British statistician, Sir Ronald Fisher , said that he favored a 5 percent threshold. So 5 percent became the Holy Grail. Unfortunately, the establishment of a 5 percent hurdle for statistical significance has had the perverse effect of encouraging researchers to do whatever…

Roadsign rusted with shotgun holes in it

The Texas Sharpshooter Fallacies

Gary Smith discusses his book, the AI Delusion, and how the pressure to publish or perish corrupts research

Bob Marks and Gary Smith offer a range of startling examples of how the pressure to publish drives a lack of rigor — and sometimes honesty — in analyzing and presenting experimental data. The result is a never ending parade of headlines in health and medicine that are unwarranted and often reversed or impossible to replicate. Shownotes 01:00 | Data…