The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis

74 studies, 56,886 K-12 students (45 elementary studies, N = 31,555; 29 secondary studies, N = 25,331); over 700 candidate reports screened, 1980-2011

K-12 mathematics, any country if reported in English but overwhelmingly US; programmes grouped as supplemental CAI (Jostens, PLATO, Larson Pre-Algebra, SRA Drill), computer-managed learning (Accelerated Math only) and comprehensive models (Cognitive Tutor, I Can Learn).

Best Evidence Encyclopedia review (Johns Hopkins CRRE, IES-funded), notable for inclusion standards that pre-empt most of the inflation this literature is famous for. Criterion 7 excluded measures "inherent to the program"; only standardized/state/district tests and comprehensive experimenter-made measures fair to controls were admitted, and the authors describe the pool as "primarily standardized tests". Criterion 5 required randomization or matching with pretest adjustment; pretest gaps > 0.5 SD were excluded; minimum duration 12 weeks; at least two teachers per arm; programmes had to be replicable in ordinary schools. Random-effects model (heterogeneity Q = 345.80, df = 73, p < .001); fixed-effects average was only +0.10. Abstract and discussion quote +0.15 while Table 2 gives the random-effects estimate as +0.16 - a small internal inconsistency in the paper, not a transcription error here. Independence: neither author develops mathematics technology; the review is unusual in that its conclusions cut against the sector it reviews.

Educational technology in K-12 mathematics moves standardized achievement +0.16 SD (random effects; +0.10 fixed) - and the entire effect is carried by weak designs. Restricting to randomized experiments halves it to +0.08; restricting to LARGE randomized experiments takes it to +0.06 (95% CI 0.00 to 0.13, p = .07), i.e. statistically indistinguishable from the +0.03 of the federal Dynarski/Campuzano RCTs. Small studies returned twice the effect of large ones (+0.26 vs +0.12) inside every design category. By programme type the ordering is the reverse of the marketing: cheap supplemental drill CAI +0.19, computer-managed learning +0.09, and the flagship comprehensive adaptive models (Cognitive Tutor, I Can Learn) +0.06 and not significant. Publication bias is NOT the explanation - published and unpublished reports both average +0.15.

Low for the randomized subset (26 studies); medium overall, since 48 of 74 studies are matched or post-hoc-matched quasi-experiments where selection into technology-using classrooms is plausible. The design moderator is itself the estimate of that contamination.

The central pattern - effects shrinking toward zero as design quality and sample size rise - replicates in the authors' own companion reading meta-analysis (Cheung & Slavin 2012, large randomized +0.07) and independently in the two federal Dynarski/Campuzano RCTs (+0.03 in math), which are themselves inside this pool. Cheung & Slavin (2016, 'How methodological features affect effect sizes in education') generalises the same design-quality gradient across subjects. No independent re-analysis of this specific study pool is known to us.