The Evidence on Teaching

Online Charter School Study 2015

Center for Research on Education Outcomes (CREDO), Stanford University; with Mathematica Policy Research and the Center on Reinventing Public Education · 2015

grade Cquasi-experimentindependentreplicated
Sample
158 full-time online charter schools across 17 states and the District of Columbia; tested online enrolment grew from ~35,000 (2009-10) to over 65,000 (2012-13); the switcher analyses use ~118,000-120,000 student-year records
Population
Students enrolled full-time in publicly funded, independently managed online ("cyber") charter schools — the closest large-scale administrative proxy the US has for full-time home-delivered schooling. Blended and part-time online programmes were deliberately excluded.
Design
Three nested designs, of increasing credibility. (1) The headline is CREDO's Virtual Control Record method: each online charter student is matched to a synthetic "virtual twin" drawn from the traditional public schools the online students left, on demographics and prior test scores. That is matching on observables and inherits the usual selection critique — which is why this file is graded C, matching the archive's treatment of the CREDO urban charter study. (2) A falsification check replaces the traditional-public comparison with brick-and-mortar CHARTER students matched the same way; results are essentially identical, which rules out "it is the charter part, not the online part". (3) The strongest piece is the "future online charter choosers" analysis: it keeps only students who would LATER enrol in an online charter, and compares their achievement in their first observed traditional-school year to their first online charter year, conditioning on prior reading and math scores plus demographics, state and grade dummies. Because the comparison is within eventual choosers, stable family selection is differenced out; what survives is the switcher-design threat that families move online in response to a shock that itself predicts decline.
Key findings
Against matched traditional-public twins, online charter attendance produced -0.25 SD in math (reported as 180 fewer days of learning) and -0.10 SD in reading (72 fewer days) per year. The effect was negative in every racial-ethnic subgroup, negative in essentially every state examined, and negative against brick-and-mortar charter comparisons as well as district comparisons. In math, 88% of online charters had weaker growth than their comparison schools; in no state did online students beat their peers in both subjects. The within-chooser analysis reproduces the result and makes it larger: enrolling in an online charter is associated with -0.17 SD in reading and -0.34 SD in math relative to the same students' own prior traditional-school year. Students who left online charters and returned to traditional schools posted second-year math growth of +0.55 SD, while those who stayed were 0.39 SD below them. The mixed-methods school-level correlations (n=60 schools with survey data) are explicitly non-causal and are not recorded here as effects.
Genetic confound
Low for the within-chooser and switcher analyses — a child's genotype does not change when their school does, and prior achievement is conditioned on. Medium-to-high for the headline VCR matching, which controls only observables. The residual threat in both is time-varying and selection-into-timing: families move a child online BECAUSE something has gone wrong (illness, bullying, a collapsing year), and that same something predicts the decline. This cuts the other way from the usual homeschooling confound — here selection plausibly makes the estimate too negative, whereas in the Rudner/Ray convenience samples it makes estimates too positive.
Replication notes
Replicated in design and direction by Fitzpatrick et al. 2020 (Indiana administrative switcher design, large persistent negatives in both subjects) and by Cordes et al. on Pennsylvania cyber charters, both of which are already in this archive. Three independent teams, three data sources, three identification strategies, same sign and similar magnitude. This is the only part of the home-delivery literature with a replicated result resting on within-student comparison.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Math growth, online charter vs matched traditional public school twinstandard deviations per year (CREDO "days of learning")-0.25 SD (equivalent to 180 fewer days of learning), p<.01standardizedannual growth, pooled 2008-09 to 2012-13business-as-usualend-of-treatmentdomain-skill
Reading growth, online charter vs matched traditional public school twinstandard deviations per year (CREDO "days of learning")-0.10 SD (72 fewer days of learning), p<.01standardizedannual growth, pooled 2008-09 to 2012-13business-as-usualend-of-treatmentdomain-skill
Reading achievement of future online charter choosers, own traditional-school year vs own first online charter yearOLS coefficient on the online-charter indicator, conditioning on prior reading and math scores, demographics, state and grade-0.17 SD (se 0.009, p<.01), n=120,376 recordsstandardizedfirst online charter year vs first observed year in the databusiness-as-usualend-of-treatmentdomain-skill
Math achievement of future online charter choosers, own traditional-school year vs own first online charter yearOLS coefficient on the online-charter indicator, same specification-0.34 SD (se 0.009, p<.01), n=118,157 recordsstandardizedfirst online charter year vs first observed year in the databusiness-as-usualend-of-treatmentdomain-skill
Math growth of students who stayed in an online charter vs those who returned to a traditional public schoolmarginal OLS coefficientleavers +0.55 SD in their second year; stayers -0.39 SD relative to leavers (p<.01, n=103,136)standardizedsecond year after initial online enrolmentactive-alternativeend-of-treatmentdomain-skill
Robustness to the charter-versus-online confoundsign agreement across comparison groupsnegative against brick-and-mortar CHARTER twins in every subgroup except Native American reading; no material differences from the district comparisonstandardizedannual growthactive-alternativeend-of-treatmentdomain-skill
Math effect by racial-ethnic groupstandard deviations per yearWhite -0.25, Black -0.22, Hispanic -0.29, Asian/PI -0.26, Native American -0.30, multi-racial -0.26 (all p<.01)standardizedannual growthbusiness-as-usualend-of-treatmentdomain-skill

Cited by