The Evidence on Teaching

Test-based promotion policies, dropping out, and juvenile crime

Eren O, Depew B, Barnes S · 2017

grade Bquasi-experimentindependentmixed
Sample
31,173 (grade 4) and 32,450 (grade 8) in the July retention sample; roughly 143,000-148,000 in the March subgroup samples; cohorts 2000-2003
Population
Louisiana fourth and eighth graders taking the LEAP English language arts and mathematics tests
Design
Journal of Public Economics 153, 9-31; full text read from the 2015 LSU working-paper version, abstract only for the published article. The design is unusually informative because it runs TWO discontinuities on the same policy. Louisiana required an 'Approaching Basic' score on both LEAP subjects; failing the March exam triggered summer remediation, and failing the July retest triggered retention. So the MARCH cutoff identifies the net policy effect - summer school plus the threat and partial realisation of retention - while the JULY cutoff identifies retention itself, conditional on having already failed in March. Very few papers in this literature can separate the remediation from the holding-back, and the two answers differ in sign.
Key findings
SEPARATING THE BUNDLE REVERSES THE SIGN. At the MARCH cutoff, eighth graders who barely miss - and therefore qualify for summer remediation - are 2 to 2.5 percentage points LESS likely to drop out, and 0.4 percentage points less likely to be convicted of a felony. At the JULY cutoff, retention ALONE raises dropout: 5.1 percentage points in grade 4 (SE 0.023, a 15% increase on a 33% control mean) and 3.3 points in grade 8 (SE 0.018, a 7% increase on a 44% mean). Retention alone has essentially no juvenile-crime effect - grade-4 estimates positive but imprecise, grade-8 estimates 'all virtually equal to zero'. The sex pattern is odd and replicated: grade-4 harm is male-only (+0.066 versus +0.002 for girls) while grade-8 harm is female-heavy (+0.041 for girls versus -0.005 for boys), the same female skew Jacob and Lefgren found in Chicago. Note the OLS bias: naive estimates are 0.094 (grade 4) and 0.155 (grade 8), roughly twice to five times the causal ones, which the authors call 'evidence for non-negligible positive selection biases'.
Genetic confound
Low at both cutoffs. Assignment is by test score with sharp thresholds applied statewide, and the July sample is further conditioned on having already failed in March, so the compliers are compared with children who failed the same March exam. The remaining caveat is that the March discontinuity identifies a policy package, not a single input.
Replication notes
This paper EXPLICITLY CONTRADICTS Jacob and Lefgren 2009 on early-grade retention: 'unlike Jacob and Lefgren (2009), we find long lasting effects of early grade retention on the propensity to drop out.' They tested and rejected the obvious reconciliation - catch-up rates are similar in the two settings (the IV effect of grade-4 retention on being in grade 8 four years later is -0.335, SE 0.018) - and conclude that 'the effect of grade retention on dropping out may depend significantly on the setting and institutions of where it occurs'. Their early-grade harm is directionally corroborated by Zhong's Texas earnings and graduation results and contradicted by Florida's precise attainment nulls.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Dropping out after grade-4 retention (July cutoff, retention alone)percentage points+5.1 (SE 2.3) instrumented, a 15% increase on a 33% control mean; reduced-form estimates 3.0-3.3; OLS 9.4administrativethrough high schoolbusiness-as-usualover-2yrattainment
Dropping out after grade-8 retention (July cutoff, retention alone)percentage points+3.3 (SE 1.8) instrumented, a 7% increase on a 44% control mean; reduced-form 2.0-2.9; OLS 15.5administrativethrough high schoolbusiness-as-usualover-2yrattainment
Dropping out after qualifying for SUMMER REMEDIATION (March cutoff, grade 8)percentage points-2 to -2.5 - barely missing the March cutoff and thus receiving remediation REDUCES dropout, the opposite sign to retention itselfadministrativethrough high schoolbusiness-as-usualover-2yrattainment
Juvenile felony conviction (March cutoff, net policy effect)percentage points-0.4, marginally significant for grade 8administrativejuvenile yearsbusiness-as-usualover-2yrbehaviour
Juvenile crime from retention alone (July cutoff)percentage pointsgrade 4 positive but imprecise; grade 8 'all virtually equal to zero'administrativejuvenile yearsbusiness-as-usualover-2yrbehaviour
Sex heterogeneity in the dropout effectpercentage pointsgrade 4: male +6.6 (significant) versus female +0.2; grade 8: female +4.1 (significant) versus male -0.5administrativethrough high schoolbusiness-as-usualover-2yrattainment

Cited by