The Evidence on Teaching

The Causal Effects of Grade Retention on Behavioral Outcomes

Martorell P, Mariano LT · 2018

grade Bquasi-experimentindependentmixed
Sample
92,425 eligible students, 76,167 in the analytic sample at the main bandwidth; cohorts 2004-2011 with outcomes to 2013-14
Population
New York City public school students in grades 3-8 near the promotion cutoff
Design
Journal of Research on Educational Effectiveness 11(2), 192-216; full text read from the IES grantee submission. Same New York City fuzzy RD as the high-school-outcomes paper, first stage 0.642-0.648, but the outcomes here are administrative discipline and attendance records rather than tests - so there is no shared-instrument problem and no incentive confound. The analysis applies a false-discovery-rate correction across the full set of behavioural outcomes, which is why so few estimates survive.
Key findings
The stigma hypothesis - that being held back damages a child's behaviour and engagement - does not survive a well-powered causal test. Pooled across grades 3 to 8: any suspension at one, two and three years -0.010, -0.001 and +0.017, all null; days suspended +0.322, +0.263 and -0.007, all null; chronic absence null throughout. THE ONLY POOLED ESTIMATE SURVIVING FDR CORRECTION IS BENEFICIAL: attendance rate +0.012 (SE 0.005) at one year and +0.010 (SE 0.004) at two. The authors' summary is the right one to record: 'we do not find evidence of systematic effects of retention on behavioral outcomes in either direction... When present, these isolated non-persistent effects tend to be beneficial when found for retained elementary school students and mixed for retained middle school students.' Only two by-grade estimates survive correction, one of them adverse (grade 7, days suspended in year two, +3.411, SE 0.865), and the authors attribute the scatter to school-transition artefacts: 'most of the instances of statistically significant estimates coincide with the transitions into and out of middle school.'
Genetic confound
Low. Sharp cutoff, very strong first stage, administrative outcomes. Conduct problems are heritable and correlate with low achievement, which is precisely why the pre-RD literature found retained children behaving worse; the discontinuity holds that correlation fixed.
Replication notes
Contradicted in part by Ozek's Florida results, which find disciplinary incidents up 30-50% for two years after third-grade retention before returning to null; corroborated by Hwang and Koedel's Indiana study, which finds discipline and absence null under both same-grade and same-age framings and notes it is 'at odds with Ozek (2015)'. Two nulls and one transient positive, in three states with different policies. Ozek's own relative-age mechanism - about a fifth to a third of his year-one effect is attributable to simply being older in the grade - narrows the gap further.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Any suspension after retention, pooled grades 3-8probability-0.010 (SE 0.012) at one year, -0.001 (0.012) at two, +0.017 (0.015) at three - all nulladministrative1-3 years after retentionbusiness-as-usualover-2yrbehaviour
Days suspended, pooled grades 3-8days+0.322 (SE 0.302), +0.263 (0.387), -0.007 (0.477) at one, two and three years - all nulladministrative1-3 years after retentionbusiness-as-usualover-2yrbehaviour
Attendance rateproportion of days+0.012 (SE 0.005) at one year and +0.010 (SE 0.004) at two - the only pooled estimate surviving false-discovery-rate correction, and it favours retentionadministrative1-2 years after retentionbusiness-as-usual1-2yrnon-cognitive
Chronic absenceprobabilitynull at every horizonadministrative1-3 years after retentionbusiness-as-usualover-2yrnon-cognitive
By-grade estimates surviving multiple-comparison correctioncounttwo of the full set: grade 7 days suspended in year two, +3.411 (SE 0.865), adverse; and the pooled attendance result, beneficial. The authors attribute the scatter to middle-school transitionsadministrative1-3 years after retentionbusiness-as-usualover-2yrbehaviour

Cited by