The Evidence on Teaching

Re-Revisiting the Marshmallow Test: A Direct Comparison of Studies by Shoda, Mischel, and Peake (1990) and Watts, Duncan, and Quan (2018)

Falk A, Kosse F, Pinger P · 2020

grade Bcritiqueindependentnot-applicable
Sample
Reanalysis of 552 children of mothers without a college degree from the NICHD study used by Watts et al., plus 10,000-replication Monte Carlo simulations
Population
United States; the SECCYD cohort.
Design
The strongest defence of the marshmallow finding, and it is a methodological rather than an evidentiary one. Two moves - a Monte Carlo simulation of the bias introduced by censoring the delay task at 7 minutes rather than 15, and a preregistered reanalysis conditioning only on PREDETERMINED variables (sex, race, log family income, mother's age at birth, mother's education, mother's vocabulary), explicitly excluding the early-ability proxies the authors regard as over-controls.
Key findings
Watts et al.'s 7-minute task censored 55% of children against 15% in the original 15-minute task. Under 55% censoring a true correlation of 0.42 attenuates to 0.349 and a true 0.57 to 0.474, neither statistically distinguishable from Watts et al.'s observed 0.30. In the reanalysis with predetermined controls only, the grade-1 achievement coefficient falls from 0.279 to 0.181 and the age-15 coefficient from 0.236 to 0.137 - a reduction of about one third rather than two thirds, and both remain significant at p < .001. The authors conclude that Watts et al. should not be read as a falsification, and that settling the question ultimately requires targeted intervention studies.
Genetic confound
This is the crux of the dispute. Falk et al. argue that controlling early cognitive ability and the home environment over-adjusts; this archive's premise is that those controls are exactly what separates a heritable proxy from a teachable lever. Both readings are defensible and the archive records the disagreement rather than resolving it.
Replication notes
Not applicable - a reanalysis. Watts and Duncan's reply concedes partial replication of the achievement correlations and maintains that the behavioural correlations are null.
DOI / URL
10.1177/0956797619861720

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Censoring bias simulationattenuated correlationtrue 0.42 attenuates to 0.349 and true 0.57 to 0.474 under 55% censoringstandardizednot applicablenonenot-applicabledomain-skill
Grade-1 achievement with predetermined controls onlystandardized coefficient0.279 falls to 0.181, p < .001standardizedgrade 1nonenot-applicabledomain-skill
Age-15 achievement with predetermined controls onlystandardized coefficient0.236 falls to 0.137, p < .001standardizedage 15nonenot-applicabledomain-skill

Cited by