Quality of Research Design Moderates Effects of Grade Retention on Achievement: A Meta-Analytic, Multilevel Analysis

22 peer-reviewed North American studies, 207 effect sizes, screened from 199 candidates published 1990 to June 2007

North American school children, kindergarten through secondary

Educational Evaluation and Policy Analysis 31(4), 480-499. Two-level meta-analytic regression in Mplus. The contribution is the coding scheme: comparison-group quality (1 = all promoted students or matching on non-academic variables such as SES; 2 = low-achieving promoted; 3 = no significant pre-retention difference in academic ability; kappa = .93) crossed with statistical control (1 = none; 2 = distal covariate; 3 = proximal achievement covariate; 4 = pre-retention measure of the outcome itself; kappa = .91). Studies came out low quality in 4 cases (18%), medium in 12 (55%), high in 6 (27%). Critically, the authors note that NO study using random assignment exists in this literature. Heterogeneity Q = 874.52, p < .001; funnel plot showed no publication bias. An overall pooled mean across all 22 studies is never printed in the paper.

The headline is a moderator, not a mean: 'the effect size for studies with the lowest design quality is -0.30, whereas the effect size for studies with medium and high design quality is 0.04' - and the 0.04 'is not practically or statistically significantly different from 0'. Design quality is a significant predictor, B = 0.341 (SE 0.05, p = .034), explaining R-squared = .395 of between-study variance; the medium-versus-high contrast is not significant (B = -0.218, p = .452), so the break is between studies that matched on prior achievement and studies that did not. The comparison-group problem is quantified separately: 15 of 22 studies (68%) used same-grade comparison and only 5 (23%) same-age, and the fade rate is about three times steeper under same-grade (-0.111 per year, p = .004) than same-age (-0.040 per year, p = .017) - the signature of an advantage that consists of being older and having covered the material twice. The authors state both halves of the honest conclusion: 'Results challenge the widely held view that retention has a negative impact on achievement' and 'the null hypothesis of no effect of retention on achievement cannot be rejected', but also that the results 'provide little support for proponents of grade retention. Given the expense... a finding of no significant difference calls into question the educational benefits of grade retention policies.' One further pattern worth recording: the four Chicago studies split by policy era, with small positive effects in the two conducted under the test-based promotion policy and moderate negative effects in the two earlier ones.

This paper IS the genetic-confound argument, made empirically and without using the word. Retention is assigned on low achievement; achievement is substantially heritable; so a study that does not match on prior achievement is comparing children who differ in the trait the study is about. Move from studies that fail that test to studies that pass it and the measured harm of retention disappears. That is exactly what the archive's premise predicts a confounded literature will do under better designs.

Reproduced structurally by Goos, Pipa and Peixoto in 2021 on a larger, later and design-gated corpus (84 studies, pooled g = -0.04). It also recovers a result already present in the 1989 Holmes meta-analysis, whose own best-matched strata - five studies matched on both achievement and SES - gave -0.01, and +0.12 with one questionable study removed. The finding has therefore been independently arrived at three times across thirty years, and quoted almost never.