Controlling, Confounding, and Construct Clarity: Responding to Criticisms of 'Revisiting the Marshmallow Test' by Doebel, Michaelson, and Munakata (2020) and Falk, Kosse, and Pinger (2020)
Watts TW, Duncan GJ · 2020
grade Dcritiqueindependentnot-applicable
Sample
Not applicable - reply article
Population
Not applicable.
Design
The reply that clarifies what is and is not in dispute, which is what makes it worth recording.
Key findings
Concedes partial replication - correlations with adolescent academic achievement were smaller than Shoda et al. reported but still positive and significant, indicating at least partial replication of the achievement correlations. Maintains that in contrast to the original, correlations with BEHAVIOURAL outcomes were null. Defends the controls as the whole point of the exercise, namely distinguishing a predictive correlation from a malleable causal lever, rejects the multicollinearity objection, and notes that Falk et al.'s confidence-interval argument cuts both ways since the original estimates rest on 35 children.
Genetic confound
The reply is explicitly about whether controlling heritable early ability is over-adjustment. This archive sides with controlling it, while recording that the question is contested.
Replication notes
Not applicable.
DOI / URL
10.1177/0956797619893606
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Status of the achievement correlations | verdict | partially replicated at smaller magnitude | standardized | age 15 | none | not-applicable | domain-skill |
| Status of the behavioural correlations | verdict | null, in contrast to the original | standardized | age 15 | none | not-applicable | behaviour |
Cited by
- Are self-control and grit teachable levers on achievement?mixedconf: highgc: medium