How features of educational technology applications affect student reading outcomes: A meta-analysis

84 studies, 60,553 K-12 participants (8 kindergarten studies, N = 2,068; 59 elementary, N = 34,200; 18 secondary, N = 24,285)

K-12 reading, any country if reported in English, overwhelmingly US. Four programme categories - supplemental CAI (Destination Reading, PLATO Focus, Waterford, WICAT), computer-managed learning (Accelerated Reader only), innovative technology applications (Fast ForWord, Reading Reels, Lightspan) and comprehensive models (READ 180, Writing to Read, Voyager Passport).

Companion to the 2013 mathematics review; same Best Evidence Encyclopedia inclusion standards. Criterion 7 excluded measures of objectives inherent to the programme and also excluded phonemic awareness, oral vocabulary and writing measures, so the pool is dominated by standardized reading tests; comprehensive experimenter-made reading measures were admitted only where fair to controls. Random assignment or pretest-adjusted matching required; pretest gaps > 0.5 SD excluded; 12-week minimum; two teachers per arm. Random effects (Q = 362.52, df = 83, p < .001). INDEPENDENCE CAVEAT, and this is why the flag is developer-involved rather than independent: the "innovative technology applications" category the review calls promising contains Reading Reels, the embedded-multimedia component of Slavin's own Success for All model, and the two qualifying Reading Reels trials (Chambers et al. 2006, 2008; ES +0.17 and +0.27) are co-authored by both reviewers. The conflict touches 2 of 84 studies and one of four programme categories; it does not touch the deflationary headline, which runs against the reviewers' interest. Version read: the April 2012 Johns Hopkins CRRE/BEE technical report ("The Effectiveness of Educational Technology Applications for Enhancing Reading Achievement in K-12 Classrooms: A Meta-Analysis"), whose abstract, N, and all reported effect sizes match the published Educational Research Review 7(3) 198-215 article; the published version's title is the one recorded above.

Educational technology moves K-12 reading +0.16 SD overall, and again the effect is a function of study quality rather than of technology: randomized experiments give +0.08, large randomized experiments +0.07, and small studies twice what large ones give (+0.25 vs +0.13). The supplementary CAI that dominates actual classroom use - the category the federal Dynarski/Campuzano RCTs evaluated - returns +0.11, which the authors call not educationally meaningful. Unlike the mathematics companion, this review DOES detect publication bias: published articles +0.25 vs technical reports and dissertations +0.14 (p < .04). The categories that look better (comprehensive integrated models such as READ 180, +0.28; innovative applications, +0.18) contain no randomized studies at all for the comprehensive models, and the authors caution that non-randomized studies overstate effects.

Medium - 59 of 84 studies are matched or post-hoc-matched quasi-experiments; only 25 are randomized. Within-study subgroup results by ability, gender and race are fixed-effects post-hoc comparisons on small k and should not be treated as causal.

The design-quality gradient (randomized +0.08, large randomized +0.07, small studies twice large studies) replicates exactly in the authors' companion mathematics meta-analysis (Cheung & Slavin 2013) and matches the federal Dynarski/Campuzano reading RCTs (+0.04 grade 1, +0.02 grade 4) that sit inside this pool. Cheung & Slavin (2016) generalises the gradient. The favourable categories (comprehensive models +0.28, innovative applications +0.18) rest on small, non-randomized study counts and are NOT replicated - the authors say so themselves.