Mind Matters Natural and Artificial Intelligence News and Analysis

TagRonald Fisher

Worried doctors and medical researchers on conference meeting, discussing possible solutions for resolving a world health crisis. Health and medical care concept. Selective focus.

Computers Excel at Finding Temporary Patterns

Which contributes to the replication crisis in science

The scientific method calls for the rigorous testing of plausible theories, ideally through randomized controlled trials. For example, a study of a COVID-19 vaccine might give the vaccine to 10,000 randomly selected people and a placebo to another 10,000, and compare the infection rates for the two groups. If the difference in the infection rates is too improbable to be explained by chance, then the difference is deemed statistically significant. How improbable? In the 1920s, the great British statistician, Sir Ronald Fisher , said that he favored a 5 percent threshold. So 5 percent became the Holy Grail. Unfortunately, the establishment of a 5 percent hurdle for statistical significance has had the perverse effect of encouraging researchers to do whatever…

Snooping Dmitry Ratushny Unsplash 1455368109333-ebc686ad6c58

Machine Learning, Part 3: Don’t Snoop on Your Data

You risk using a feature for prediction that is common to the dataset, but not to the problem you are studying

As long as we can establish that our theories, hypotheses, and/or models are independent of the data, then we can trust that their predictive power will generalize beyond the data we have observed.

Read More ›