The Evidence on Teaching

Controlling, Confounding, and Construct Clarity: Responding to Criticisms of 'Revisiting the Marshmallow Test' by Doebel, Michaelson, and Munakata (2020) and Falk, Kosse, and Pinger (2020)

Watts TW, Duncan GJ · 2020

grade Dcritiqueindependentnot-applicable
Sample
Not applicable - reply article
Population
Not applicable.
Design
The reply that clarifies what is and is not in dispute, which is what makes it worth recording.
Key findings
Concedes partial replication - correlations with adolescent academic achievement were smaller than Shoda et al. reported but still positive and significant, indicating at least partial replication of the achievement correlations. Maintains that in contrast to the original, correlations with BEHAVIOURAL outcomes were null. Defends the controls as the whole point of the exercise, namely distinguishing a predictive correlation from a malleable causal lever, rejects the multicollinearity objection, and notes that Falk et al.'s confidence-interval argument cuts both ways since the original estimates rest on 35 children.
Genetic confound
The reply is explicitly about whether controlling heritable early ability is over-adjustment. This archive sides with controlling it, while recording that the question is contested.
Replication notes
Not applicable.
DOI / URL
10.1177/0956797619893606

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Status of the achievement correlationsverdictpartially replicated at smaller magnitudestandardizedage 15nonenot-applicabledomain-skill
Status of the behavioural correlationsverdictnull, in contrast to the originalstandardizedage 15nonenot-applicablebehaviour

Cited by