Assessing the impact of grade retention: A cautionary tale of exclusion restriction violations

Two cohorts of Michigan students subject to the state's third-grade reading law

Michigan third graders flagged for retention under the Read by Grade Three law

Journal of Public Economics 253, 105534; full abstract read verbatim, plus the full text of the working-paper version (EdWorkingPaper 25-1188, April 2025, titled 'The Impacts of Grade Retention Policy With Minimal Retention'). This is the methodological attack that lands on the entire regression-discontinuity retention literature, and its author list matters: Brian Jacob wrote the founding RD study of this literature in 2004. The argument is about the EXCLUSION RESTRICTION. RD studies scale a reduced-form effect of falling below the cutoff by the jump in retention probability to recover a local average treatment effect. That step assumes falling below the cutoff affects outcomes ONLY through retention. If flagged-but-promoted students also receive intensive support, the assumption fails and the scaled estimate is inflated by roughly one over the first stage.

In Michigan, being flagged for retention raises reading performance by about 0.05 SD - but raises the probability of actually being retained by only 3.4 percentage points, 'so the implied effect of retention itself under standard assumptions would be an implausibly large 1.3 SD'. Two pieces of evidence show why: survey data indicate that flagged students receive more intensive reading support even when they are not retained, and 'we estimate similar effects in districts that did not retain any students' - a placebo that the exclusion restriction cannot survive. The authors name the affected papers (Mariano and colleagues 2024, Mumma and Winters 2023, Ozek 2015, Winters and Greene 2012) and state the implication precisely: 'the prior literature did not seriously contemplate that being flagged for retention might influence student learning even in the absence of actual retention... LATE estimates (but not ITT estimates) in the prior literature might be biased upward.' THE SIZE OF THE BIAS SCALES INVERSELY WITH THE FIRST STAGE, which gives a clean ranking of exposure: Michigan 3.4 points (catastrophic), Mississippi 5.8 points (severe - a 0.067 SD reduced form becomes a 1.15 SD LATE), Florida 28-32 points (moderate), Texas 35.5 points (moderate), Uruguay about 43 points (small), New York City about 64 points (small, and pre-empted by that paper's authors).

Not the subject. The critique concerns instrument validity, not selection on ability; the reduced-form estimates it defends are as free of genetic confounding as any RD in the archive. Its practical effect on this topic is to demote instrumented magnitudes to reduced-form ones, which shrinks every headline number without changing any sign.

The critique is itself a vindication of an earlier one. Joseph Cimpian made substantially this argument against Schwerdt, West and Winters in a 2015 NEPC review - noting that Florida's law requires intensive services for everyone below the cut score, retained or not, and that the instrumented scaling 'attributes the entire above/below-threshold difference to just retained students, effectively making the outcome difference appear more than three times as large' - and was widely dismissed as advocacy. A decade later the same objection arrived with survey evidence and a zero-retention placebo, in a top field journal, from the literature's founder.