Teaching Children to Read: The Fragile Link Between Science & Federal Education Policy
Camilli G, Vargas S, Yurecko M · 2003
grade Ccritiqueindependentmixednumbers spot-checked
Sample
40 studies / 225 comparisons (the National Reading Panel corpus)
Population
Same K-6 corpus the NRP used.
Design
Recomputes the NRP phonics effect sizes coding BOTH treatment and control groups for degree of systematic phonics, plus tutoring, language activities and outcome-measure type, using weighted multiple regression. The NRP had compared systematic phonics against a control that often contained phonics too, which inflates the contrast.
Key findings
The strongest technical critique of the NRP phonics number. Once the control group's own phonics is coded, the systematic-versus-some-phonics advantage is ~0.24-0.27 rather than 0.41, and tutoring (0.40) and language activities (0.21-0.29) match or beat it. IMPORTANT correction to a common claim: within this corpus standardized tests gave LARGER effects (+0.186), so the '2x researcher-measure inflation' rule does NOT apply to the phonics word-reading estimate (it does apply to comprehension and vocabulary).
Genetic confound
Experimental corpus, so passive gene-environment correlation is not the issue; the confound at stake is instructional co-intervention (tutoring, language activities) riding along with phonics.
Replication notes
Extended by the same team's 2006 multilevel reanalysis; rebutted on estimand grounds by Stuebing et al. 2008 and defended in Camilli, Kim & Vargas 2008. Load-bearing coefficients checked against the EPAA full text.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Systematic vs SOME phonics (weighted regression) | d | 0.24 (WGT1) / 0.188 (WGT3); simple contrast 0.271 — roughly half the NRP's 0.41 | mixed | end-of-treatment | active-alternative | end-of-treatment | domain-skill |
| Individual tutoring (moderator) | d | 0.40 — equal to or larger than systematic phonics itself | mixed | end-of-treatment | active-alternative | end-of-treatment | domain-skill |
| Language-rich activities in the treatment condition | d | 0.21-0.29 | mixed | end-of-treatment | active-alternative | end-of-treatment | domain-skill |
| Studies using standardized tests | d | +0.186 — standardized tests gave LARGER effects here, not smaller | standardized | end-of-treatment | active-alternative | end-of-treatment | domain-skill |
Cited by
- Systematic/explicit phonics vs whole language and balanced literacystrong supportconf: highgc: low