You’ve maybe heard of Stanford University’s “marshmallow experiment,” right? A child’s future can be predicted, we were told by psychologist Walter Mischel (1930–2018), by whether the child can delay gratification:

Walter Mischel’s pioneering research at Bing in the late 1960s and early 1970s famously explored what enabled preschool-aged children to forgo immediate gratification in exchange for a larger but delayed reward…

This research identified some of the key cognitive skills, strategies, plans and mindsets that enable self-control. If the children focused on the “hot” qualities of the temptations (e.g., “The marshmallows are sweet, chewy, yummy”), they soon rang the bell to bring the researcher back. If they focused on their abstract “cool” features (“The marshmallows are puffy and round like cotton balls”), they managed to wait longer than the researchers, watching them through a one-way observation window, could bear. And when they imagined that the treats facing them were “just a picture” and were cued to “put a frame around it in your head” they were able to wait for almost 18 minutes. When Mischel asked a child how she managed to wait so long, she replied: “well you can’t eat a picture.”

Janine Zacharia, “The Bing “Marshmallow Studies”: 50 Years of Continuing Research” at Stanford University (September 24, 2015)

The larger significance, we were told, was yet to come:

The Bing research also yielded a surprise: What the preschoolers did as they tried to wait, unexpectedly predicted much about their future lives. “The more seconds they waited at age 4 or 5, the higher their SAT scores and the better their rated social and cognitive function in adolescence,” Mischel writes in his recent book, The Marshmallow Test: Mastering Self-Control.

Children who waited longer tended to become more self-reliant, more self-confident, less distractible and more able to cope with stress as adolescents, he said. But, he added, he has reassured anxious parents over the years that a child’s ability to delay gratification in preschool does not determine their future. “Clearly, your future is not in a marshmallow,” he said, debunking the pithy but incorrect way popular media have summed up his findings.

Janine Zacharia, “The Bing “Marshmallow Studies”: 50 Years of Continuing Research” at Stanford University (September 24, 2015)

Actually, the media were not necessarily wrong. The researchers were studying the mechanisms underlying self-control and parents and teachers used the findings to encourage children to develop the habit of thinking before acting.

The study was very influential:

Parents devised their own marshmallow tests. Success gurus gave TED Talks about it. Sesame Street’s notoriously out-of-control Cookie Monster starred in a series of videos demonstrating delay of gratification skills he learned from Mischel.

In 2015, the Bing project scientists won the 2015 Golden Goose Award, bestowed by a group of congressmen for government funded research that leads to significant public benefits.

Dee Gill, “New Study Disavows Marshmallow Test’s Predictive Powers” at UCLA Anderson Review (February 24, 2021) The paper is open access.

But all good things can go overboard. As with so many studies, a lot depended on the children’s overall circumstances:

More recent research addressed many of the problems with Mischel’s study design including treat choice, sample size, and the population from which the sample was drawn. “The updated version of the marshmallow test – in which the children were able to choose their own treats, including chocolate – studied 900 children, with the sample adjusted to make it more reflective of US society, including 500 whose mothers had not gone on to higher education. Mischel’s original research used children of Stanford University staff, while the followup study included fewer than 50 children from which Mischel and colleagues formed their conclusions.”

This newer research failed to replicate Mischel’s results and showed that the test has very little predictive value at all. Tyler Watts, one of the researchers, had this to say, “‘We found virtually no correlation between performance on the marshmallow test and a host of adolescent behavioural outcomes.’”2 The test might show how well a child can delay gratification now, but it doesn’t show how well they’ll be able to do it in the future.

Chris Loper, “The “Marshmallow Test” Was Debunked… Here’s Why That Matters” at Northwest Educational Services (May 4, 2020)

One issue Loper raises is that the prefrontal cortex, thought to play a role in self-monitoring, is not very well-developed for that purpose in young children. Its performance cannot be used as a predictor for later performance. Put another way, self-monitoring and self-restraint take many years of practice so using the study as a predictor (which was not really intended by the authors) is not recommended.

Marshmallow experiments proliferated anyhow:

A more recent study at UCLA Anderson found pretty much the same thing as Loper notes:

Following the Bing children into their 40s, the new study finds that kids who quickly gave in to the marshmallow temptation are generally no more or less financially secure, educated or physically healthy than their more patient peers. The amount of time the child waited to eat the treat failed to forecast roughly a dozen adult outcomes the researchers tested, including net worth, social standing, high interest-rate debt, diet and exercise habits, smoking, procrastination tendencies and preventative dental care, according to the study published in the Journal of Economic Behavior and Organization.

“With the marshmallow waiting times, we found no statistically meaningful relationships with any of the outcomes that we studied,” UCLA Anderson’s Daniel Benjamin, who brings expertise to the study that includes behavioral economics and statistical methodology, says in an interview.

Dee Gill, “New Study Disavows Marshmallow Test’s Predictive Powers” at UCLA Anderson Review (February 24, 2021) The paper is open access.

One problem with the marshmallow test is that it assumes that the child believes that he will in fact receive a better reward later and not all children have reason to believe that:

In 2018, a major marshmallow test study gained fame for failing to find strong correlations between wait times and adolescent outcomes. Published in Psychological Science and led by Tyler W. Watts (now at Columbia University), the study followed a much more diverse group of 900 preschoolers into their teens. Controlling for differences such as household income and cognitive abilities, they found only weak relationships to academic outcomes and no significant correlations to later behaviors, such as anti-social tendencies.

The Watts study findings support a common criticism of the marshmallow test: that waiting out temptation for a later reward is largely a middle or upper class behavior. If you come from a place of shortages and broken promises, eating the treat in front of you now might be the better bet than trusting there will be more later.

Dee Gill, “New Study Disavows Marshmallow Test’s Predictive Powers” at UCLA Anderson Review (February 24, 2021) The paper is open access.

The Anderson study, which preregistered its longterm study with Open Science Framework to help prevent researcher bias in interpretation, did come up with some useful information:

They identified 11 measures of capital formation, including credit card misuse, high-rate debt, income, social status and financial security, as well as diet, smoking and alcohol habits, weight and procrastination tendencies.

As the researchers predicted, the study finds only a tiny correlation between marshmallow test times and midlife capital formation. A graduate’s score on the self-regulation index was, however, modestly predictive of their middle-age capital formation, the study finds.

Dee Gill, “New Study Disavows Marshmallow Test’s Predictive Powers” at UCLA Anderson Review (February 24, 2021) The paper is open access.

In short, whether a child could wait for a second marshmallow did not predict longterm success as typically measured. As with so many efforts we make to predict the future and plan for a better one, life turns out to be trickier than we expect.

You may also wish to read these skeptical takes on data by business prof Gary Smith:

Detecting BS data: If it’s too good to be true, it probably is. A recent Wall Street Journal article shows a near-perfect link between inflation and money. But a link that near-perfect raises suspicions. Precise predictions of human behavior are implausible and rarely match up to the far more complicated reality.

The British Medical Journal’s top picks in junk medical science. In its legendary Christmas edition, the Journal highlights the worst offenders. The publicized studies have twisted and tortured data to come up with suspicious or ridiculous conclusions. Here’s how they did it.

and

Female hurricanes: How a mass of hot air became a zombie study. When a reporter first asked me about a study claiming that “Female Hurricanes are Deadlier than Male Hurricanes,” I was sceptical. Do sexist humans die because they don’t take hurricanes with female names seriously? No, the study is seriously flawed.