Educational effects of the Tools of the Mind curriculum: A randomized trial
Barnett WS, Jung K, Yarosz DJ, Thomas J, Hornbeck A, Stechuk R, Burns S · 2008
grade Crctdeveloper-involvedfailed
Sample
One urban New Jersey Abbott preschool; 85 children in 7 Tools classrooms versus 117 children in 11 comparison classrooms (the paper reports 88 versus 122)
Population
United States; 3- and 4-year-olds in a single urban preschool.
Design
The developer-involved single-site trial on which the Tools of the Mind reputation was built. Teachers and assistants were randomly assigned to classrooms and children then randomly assigned to classrooms; the comparison arm was the district's regular curriculum. One site, no blinding reported. The What Works Clearinghouse reviewed it and recomputed the effects with clustering handled properly.
Key findings
The authors reported a significant positive effect on receptive vocabulary in regression but NOT in the hierarchical model once classroom clustering was accounted for. WWC-computed standardized effects were PPVT-III g = 0.21 (not significant), EOWPVT-R g = 0.09 (not significant), oral-language average 0.15 (not significant), with no significant effects on print knowledge, cognition or maths. The surviving positive claim rests on a teacher-rated problem-behaviour scale rather than on any standardized EF task.
Genetic confound
Low as designed; the threats are single-site scope, developer involvement and clustering.
Replication notes
Failed. Two large independent evaluations - Farran and Wilson's multi-district Tennessee and North Carolina trial and Solomon et al.'s registered Canadian trial with an active control - found nulls or effects favouring the control group.
DOI / URL
10.1016/j.ecresq.2008.03.001
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Receptive vocabulary (PPVT-III) | Hedges g | 0.21, not significant once clustering handled | standardized | end of preschool year | active-alternative | end-of-treatment | domain-skill |
| Expressive vocabulary (EOWPVT-R) | Hedges g | 0.09, not significant | standardized | end of preschool year | active-alternative | end-of-treatment | domain-skill |
| Print knowledge, cognition, mathematics | significance | no significant effects | standardized | end of preschool year | active-alternative | end-of-treatment | domain-skill |
| Teacher-rated problem behaviour | direction | the paper's surviving positive claim | researcher-designed | end of preschool year | active-alternative | end-of-treatment | behaviour |
Cited by
- Can executive function be trained, and does it transfer to learning?no effectconf: highgc: low