Effects of Dual-Language Immersion Programs on Student Achievement: Evidence From Lottery Data
Steele JL, Slater RO, Zamarro G, Miller T, Li J, Burkhauser S, Bacon M · 2017
grade Bnatural-experimentindependentunreplicated
Sample
seven cohorts of dual-language immersion lottery applicants in a large urban district (Portland, Oregon)
Population
Kindergarten applicants to oversubscribed dual-language immersion programmes in Portland Public Schools (Spanish, Japanese, Mandarin, Russian, Vietnamese partner languages), followed through grade 8; both English-proficient students and English learners
Design
The identification is a genuine admissions LOTTERY among oversubscribed immersion programmes, which is what makes this the only grade-B evidence in the immersion literature: parents self-select into APPLYING, but not into winning. Intent-to-treat estimates are the headline; compliance is imperfect in both directions (lottery losers sometimes enrol later, winners sometimes decline), so treatment-on-treated effects are larger. CRITICAL SCOPE LIMIT: every achievement outcome is a state test administered IN ENGLISH. The study does not measure partner-language proficiency at all, so it cannot say how well immersion students learn the second language - the outcome most people buy immersion for.
Key findings
Positive intent-to-treat effects on ENGLISH READING of 13% to 22% of a standard deviation at grades 5 and 8 - the authors gloss this as 7 to 9 months of learning. Little benefit in mathematics and science, but also no detriment, which is the finding that answers the usual objection (that teaching half the day in another language costs content learning). By grades 6 and 7, lottery winners who had been classified as English learners were 3 to 4 percentage points less likely to remain classified as EL, and that effect was stronger when the student's native language matched the programme's partner language.
Genetic confound
Low, by design. Winners and losers are drawn from the same pool of applicant families, so the expected distribution of heritable ability and of family motivation is identical across arms. This is the whole reason the study is worth more than the rest of the literature combined: everywhere else, immersion families are the confound.
Replication notes
No second immersion lottery of comparable scale has been published, so this is a single unreplicated grade-B result carrying an entire literature. Its English-reading finding is broadly consistent in direction with the bilingual-education meta-analyses, which are much weaker designs.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| ENGLISH reading achievement, grade 5 (state test) | ITT effect size | 0.13-0.22 SD across grades 5 and 8 (glossed as 7-9 months of learning) | standardized | grade 5 | business-as-usual | over-2yr | domain-skill |
| ENGLISH reading achievement, grade 8 (state test) | ITT effect size | 0.13-0.22 SD across grades 5 and 8 | standardized | grade 8 | business-as-usual | over-2yr | domain-skill |
| Mathematics and science achievement (state tests, in English) | ITT effect size | little benefit and no detriment | standardized | grades 5 and 8 | business-as-usual | over-2yr | domain-skill |
| Probability of remaining classified as an English learner, grades 6-7 | ITT effect, percentage points | 3-4 percentage points lower for lottery winners; stronger when home language matches the partner language | administrative | grades 6-7 | business-as-usual | over-2yr | attainment |
| Partner-language (L2) proficiency | not measured | not measured by this study | unknown | not applicable | unclear | not-applicable | unclear |
Cited by
- Does dual-language immersion work, and is the gain in the second language or in English?moderate supportconf: mediumgc: low