Online Charter School Study 2015
Center for Research on Education Outcomes (CREDO), Stanford University; with Mathematica Policy Research and the Center on Reinventing Public Education · 2015
grade Cquasi-experimentindependentreplicated
Sample
158 full-time online charter schools across 17 states and the District of Columbia; tested online enrolment grew from ~35,000 (2009-10) to over 65,000 (2012-13); the switcher analyses use ~118,000-120,000 student-year records
Population
Students enrolled full-time in publicly funded, independently managed online ("cyber") charter schools — the closest large-scale administrative proxy the US has for full-time home-delivered schooling. Blended and part-time online programmes were deliberately excluded.
Design
Three nested designs, of increasing credibility. (1) The headline is CREDO's Virtual Control Record method: each online charter student is matched to a synthetic "virtual twin" drawn from the traditional public schools the online students left, on demographics and prior test scores. That is matching on observables and inherits the usual selection critique — which is why this file is graded C, matching the archive's treatment of the CREDO urban charter study. (2) A falsification check replaces the traditional-public comparison with brick-and-mortar CHARTER students matched the same way; results are essentially identical, which rules out "it is the charter part, not the online part". (3) The strongest piece is the "future online charter choosers" analysis: it keeps only students who would LATER enrol in an online charter, and compares their achievement in their first observed traditional-school year to their first online charter year, conditioning on prior reading and math scores plus demographics, state and grade dummies. Because the comparison is within eventual choosers, stable family selection is differenced out; what survives is the switcher-design threat that families move online in response to a shock that itself predicts decline.
Key findings
Against matched traditional-public twins, online charter attendance produced -0.25 SD in math (reported as 180 fewer days of learning) and -0.10 SD in reading (72 fewer days) per year. The effect was negative in every racial-ethnic subgroup, negative in essentially every state examined, and negative against brick-and-mortar charter comparisons as well as district comparisons. In math, 88% of online charters had weaker growth than their comparison schools; in no state did online students beat their peers in both subjects. The within-chooser analysis reproduces the result and makes it larger: enrolling in an online charter is associated with -0.17 SD in reading and -0.34 SD in math relative to the same students' own prior traditional-school year. Students who left online charters and returned to traditional schools posted second-year math growth of +0.55 SD, while those who stayed were 0.39 SD below them. The mixed-methods school-level correlations (n=60 schools with survey data) are explicitly non-causal and are not recorded here as effects.
Genetic confound
Low for the within-chooser and switcher analyses — a child's genotype does not change when their school does, and prior achievement is conditioned on. Medium-to-high for the headline VCR matching, which controls only observables. The residual threat in both is time-varying and selection-into-timing: families move a child online BECAUSE something has gone wrong (illness, bullying, a collapsing year), and that same something predicts the decline. This cuts the other way from the usual homeschooling confound — here selection plausibly makes the estimate too negative, whereas in the Rudner/Ray convenience samples it makes estimates too positive.
Replication notes
Replicated in design and direction by Fitzpatrick et al. 2020 (Indiana administrative switcher design, large persistent negatives in both subjects) and by Cordes et al. on Pennsylvania cyber charters, both of which are already in this archive. Three independent teams, three data sources, three identification strategies, same sign and similar magnitude. This is the only part of the home-delivery literature with a replicated result resting on within-student comparison.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Math growth, online charter vs matched traditional public school twin | standard deviations per year (CREDO "days of learning") | -0.25 SD (equivalent to 180 fewer days of learning), p<.01 | standardized | annual growth, pooled 2008-09 to 2012-13 | business-as-usual | end-of-treatment | domain-skill |
| Reading growth, online charter vs matched traditional public school twin | standard deviations per year (CREDO "days of learning") | -0.10 SD (72 fewer days of learning), p<.01 | standardized | annual growth, pooled 2008-09 to 2012-13 | business-as-usual | end-of-treatment | domain-skill |
| Reading achievement of future online charter choosers, own traditional-school year vs own first online charter year | OLS coefficient on the online-charter indicator, conditioning on prior reading and math scores, demographics, state and grade | -0.17 SD (se 0.009, p<.01), n=120,376 records | standardized | first online charter year vs first observed year in the data | business-as-usual | end-of-treatment | domain-skill |
| Math achievement of future online charter choosers, own traditional-school year vs own first online charter year | OLS coefficient on the online-charter indicator, same specification | -0.34 SD (se 0.009, p<.01), n=118,157 records | standardized | first online charter year vs first observed year in the data | business-as-usual | end-of-treatment | domain-skill |
| Math growth of students who stayed in an online charter vs those who returned to a traditional public school | marginal OLS coefficient | leavers +0.55 SD in their second year; stayers -0.39 SD relative to leavers (p<.01, n=103,136) | standardized | second year after initial online enrolment | active-alternative | end-of-treatment | domain-skill |
| Robustness to the charter-versus-online confound | sign agreement across comparison groups | negative against brick-and-mortar CHARTER twins in every subgroup except Native American reading; no material differences from the district comparison | standardized | annual growth | active-alternative | end-of-treatment | domain-skill |
| Math effect by racial-ethnic group | standard deviations per year | White -0.25, Black -0.22, Hispanic -0.29, Asian/PI -0.26, Native American -0.30, multi-racial -0.26 (all p<.01) | standardized | annual growth | business-as-usual | end-of-treatment | domain-skill |
Cited by
- Does homeschooling raise achievement and attainment, or does it select the children who would have done well anyway?insufficientconf: lowgc: high