Mind Matters Natural and Artificial Intelligence News and Analysis
xray of brain.jpg
X-ray.

Why a “Budding” Neuroscientist Is Skeptical of Brain Scans

After reading her perceptive essay about the problems in fMRI imaging in neuroscience, I’m sad that a gifted student has doubts about a career in the field

Kelsey Ichikawa has just published a superb essay about the pitfalls of functional magnetic resonance imaging (fMRI) of the brain. Ms. Ichikawa (pictured), who describes herself as a ”budding” neuroscientist who graduated last year from Harvard, discusses the snares into which misinterpretation can lead us.

fMRI brain scanning is a relatively new technology in which researchers and clinicians use magnetic resonance images (MRI) of the brain to detect brain activity almost as it happens. The technique is widely used, both for clinical care of patients (neurosurgeons use it to map sensitive parts of the brain prior to surgery) and for research purposes. A major thrust of neuroscience research in the last couple of decades has been the use of fMRI to correlate brain activity with thinking and to draw conclusions about the physical basis of the mind.

A few points about fMRI imaging are important to note.

1) fMRI imaging doesn’t see brain activity directly. fMRI imaging detects changes in regional blood flow in the brain, and we know from research over a century ago that activity in a part of the brain correlates more or less with changes in blood flow to that brain part. When neurons in a region of the brain become active, blood flow in that region increases.

2) The changes in blood flow do not occur simultaneously with the brain activity. There is a lag of anywhere from a few seconds to upwards of a minute from the neuronal activity to the uptick in blood flow. The time resolution of fMRI imaging for brain activity is not particularly good.

3) fMRI imaging produces rather fuzzy pictures of the brain — the spatial resolution of fMRI, compared with ordinary MRI, is rather poor, although it is improving.

Furthermore, fMRI requires a lot of signal-processing, which means that researchers must make choices about which data points are important and which are noise. Such decisions inherently introduce bias into the research. The researchers’ processing “smudges” the images, making interpretation considerably more difficult and unreliable.

Ichikawa gives an example of the imprecision and potential for bias in fMRI imaging:

The most common analysis procedure in fMRI experiments, null hypothesis tests, require that the researcher designate a statistical threshold. Picking statistical thresholds determines what counts as a significant voxel—which voxels end up colored cherry red or lemon yellow. Statistical thresholds make the difference between a meaningful result published in prestigious journals like Nature or Science, and a null result shoved into the proverbial file drawer.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

This opens the door for data manipulation that, while not deliberately deceptive, can seriously skew results:

Scientists are under tremendous pressure to publish positive results, especially given the hypercompetitive academic job market that fixates on publication record as a measure of scientific achievement (though the reproducibility crisis has brought attention to the detriments of this incentive structure). If an fMRI study ends up with a null or lackluster result, you can’t always go back and run another version of the study. MRI experiments are very expensive and time-intensive… You can see how a researcher might be tempted, even subconsciously, to play around with the analysis parameters just one more time to see if they can find a significant effect in the data it cost so much to obtain.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

Ichikawa quotes Rick Born, a neurobiologist at Harvard: “fMRI is clearly not pure noise, it’s a real signal, but it’s subject to many degrees of freedom, fiddling around with the data, filtering it in different ways until you can see whatever you want to see.”

She points to one of the fundamental problems with fMRI: statistical excess:

The problem of statistical excess, called multiple comparisons, looms large over this part of the analysis… Multiple comparisons means too many statistical tests. The problem of multiple comparisons is like surveying 100,000 strangers about whether they know Beyoncé personally. None of those 100,000 people are actually acquainted with her, but for each person you ask, there is a 5 percent chance they will lie and say they are, just for kicks. In the end, you tally 5,000 friends of Beyoncé, even though the ground truth is that zero of those people are friends with her. If you had asked 100 strangers, you would only end up with five incorrect measurements, but because of sheer numbers and the probability of random deception, surveying 100,000 strangers results in 5,000 incorrect measurements.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

fMRI data is often unreliable in this same way:

One person’s brain data has hundreds of thousands of voxels. By the sheer number of voxels and random noise, a researcher who performs a statistical test at every voxel will almost certainly find significant effects where there isn’t really one.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

Ichikawa points to a famous fMRI study in which researchers, using standard statistical methods, found brain activity in a dead salmon:

[I]n 2009 when an fMRI scan detected something fishy in a dead salmon. Craig Bennett, then a postdoctoral researcher at the University of California, Santa Barbara, wanted to test how far he could push the envelope with analysis. He slid a single Atlantic salmon into an MRI scanner, showed it pictures of emotional scenarios, and then followed typical pre-processing and statistical analysis procedures. Lo and behold, the dead fish’s brain exhibited increased activity for emotional images—implying a sensitive, if not alive, salmon. Even in a dead salmon’s brain, the MRI scanner detected enough noise that some voxels exhibited statistically significant correlations. By failing to correct for multiple comparisons, Bennett and his colleagues “discovered” illusory brain activity.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

Ichikawa does a great job of pointing out the myriad pitfalls of fMRI research, which is one of the most active lines of research in 21st century neuroscience. If we consider the unreliability of the technique, the inescapable opportunities for unconscious bias in data processing, and the profound conceptual confusions that are endemic in modern neuroscience, we see that neuroscientific research based on fMRI imaging is not far from 19th century phrenology. Phrenologists used bumps on the skull to infer states of the brain. Much modern fMRI research uses “bumps” in the data to infer states of the brain. While that is more technologically advanced than the use of skull bumps, it relies on the same materialist assumptions of phrenology. Many scientists and philosophers have rightly called fMRI imaging “the new phrenology.”

We must be very careful about claims about the mind-brain relationship that we hear in science media. A combination of unreliable technology, biased statistical handling of data, and the conceptual mess inherent to materialist science make meaningful claims by materialists and by neuroscientists working from an implicitly materialist perspective about the relationship between the mind and the brain next to worthless.

Ichikawa concludes:

The problem lies in what we ask and expect of these scientific results, and the authority we give them. After all, the phrase “the brain lights up” is an artifact of the images that we craft. The eye-catching blobs and connectivity maps exist because of the particular way in which neuroscientists, magnetic resonance physicists, and data scientists decided to visualize and represent data from the brain.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

She also tells us,

Now I’m questioning whether I want to continue in this fraught field… Still, I keep my Google bookmark folder for cognitive science Ph.D. programs. I attend neuroscience lectures when I can. Maybe I’ll return, either as a scientist or sociologist of neuroimaging techniques. In opening the black box of the MRI machine, I may have fallen out of love with neuroimages, but at least now I see them for what they are.

Kelsey Ichikawa, “The Trouble with Brain Scans” at Nautilus

I’m sad that Ichikawa may not pursue her career in neuroscience. We desperately need neuroscientists like her who are genuine skeptics, in the sense that genuine skeptics always are the harshest critics of their own research. We need neuroscientists who are incessantly critical of their own methods and of their own interpretations, which are almost invariably materialist interpretations. I fear however that the scientists drawn to fields like fMRI research are precisely those who blithely accept massaged data and manipulated interpretations that fit the unreflective and pervasive philosophical bias — the materialist bias — in modern neuroscience.


You may also wish to read: Why are some scientists turning away from brain scans? Sometimes, brain scans just sound like popular opinion. What’s wrong?

and

Brain scans can read your mind — in a dozen conflicting ways. A recent study involving 70 research groups identified sharp limitations in the value of brain imaging (fMRI) in understanding the mind.


Michael Egnor

Senior Fellow, Center for Natural & Artificial Intelligence
Michael R. Egnor, MD, is a Professor of Neurosurgery and Pediatrics at State University of New York, Stony Brook, has served as the Director of Pediatric Neurosurgery, and award-winning brain surgeon. He was named one of New York’s best doctors by the New York Magazine in 2005. He received his medical education at Columbia University College of Physicians and Surgeons and completed his residency at Jackson Memorial Hospital. His research on hydrocephalus has been published in journals including Journal of Neurosurgery, Pediatrics, and Cerebrospinal Fluid Research. He is on the Scientific Advisory Board of the Hydrocephalus Association in the United States and has lectured extensively throughout the United States and Europe.

Why a “Budding” Neuroscientist Is Skeptical of Brain Scans