A Multisite Randomized Study of an Online Learning Approach to High School Credit Recovery: Effects on Student Experiences and Proximal Outcomes

1,737 students in the analytic samples - 613 students in 28 Algebra 1 classes across 13 high schools, and 1,124 students in 70 English 9 classes across 19 high schools (1,683 randomised across 24 high schools in total); summer terms 2018 and 2019.

US students, roughly age 14-15, in a large urban district (Los Angeles) who had failed Algebra 1 or ninth-grade English in their first year of high school. Well below average in eighth-grade English and maths and below a C average in ninth grade. English learners below ELD level 4 were excluded by district policy, which is itself a signal about who online courses are considered unsuitable for. Inside the archive's 4-18 age band.

WHAT "ONLINE" MEANS HERE, AND WHY IT DIFFERS FROM CHICAGO: Edgenuity supplied the core online curriculum for every online class, but there was NO remote online instructor - the school provided a subject-appropriate CREDENTIALED TEACHER in the room who was expected to supplement the digital content with additional instruction and had flexibility to do so. That is the opposite of the Chicago model, where a remote online teacher held the instructional role and the person in the room was a "mentor" with no teaching expectation. Both arms met 2.5 hours a day in a standard classroom for a five-week summer session, both were expected to cover the same district content standards, and both were led by a credentialed teacher - so this is an active-alternative contrast that varies the CURRICULUM MEDIUM while holding the in-room teacher constant, whereas Chicago varied the teacher. DESIGN: student-level random assignment within blocks defined by subject, cohort and school (and in some schools by which semesters were failed); half the classes in each school were online. Baseline equivalence achieved (standardised mean differences all below 0.25). Analysis includes randomisation-block fixed effects. Registered on the Registry of Efficacy and Effectiveness Studies (REES ID 1917), and the WWC has reviewed it. ATTRITION IS THE MAIN WEAKNESS AND THE AUTHORS SAY SO: response on the study-administered survey and test was 66% for Algebra 1 (59-60% online vs 72-73% teacher-directed) and 59% for English 9 - differential by arm, and the paper states the Algebra 1 sample has high attrition under WWC thresholds even on the optimistic boundary and English 9 has high attrition on the cautious boundary. Observed characteristics of respondents were similar (SMD below 0.25) and results are robust to multiple imputation vs listwise deletion, but the learning estimates should be read with that discount. The credit-recovery outcome comes from administrative course grades and does not carry the same problem. GRADING REGIME MOVED, AND THE AUTHORS TRACE IT: the large English 9 penalty is attributed largely to unintended shifts in grading criteria - online English teachers reported tests and quizzes accounting for 67% of the final grade against 40% in the teacher-directed classes. So the biggest negative number in this study is partly a measurement artefact of how online courseware forces grading, which is itself a real consequence of adopting the product but is not the same thing as students learning less. EQUIVALENCE TESTING IS USED PROPERLY: the paper refuses to call the non-significant algebra results "equivalent", because the 90% confidence interval extends past a plausible equivalence threshold. FUNDING: IES; conducted by AIR. Edgenuity supplied the courseware, not the evaluation - independent under the archive's rule. CITATION NOTE: published online in 2023 (DOI 10.1080/19345747.2023.2198524) and assigned to Journal of Research on Educational Effectiveness 17(3), 2024; the year recorded here follows the 2023 online publication and the ERIC record (ED647309), which is the full text read.

In 24 Los Angeles high schools, replacing the summer credit-recovery curriculum with Edgenuity courseware while keeping a credentialed teacher in the room produced essentially nothing for Algebra 1 - credit recovery 62% online vs 69% teacher-directed, a model-estimated -6 percentage points (about -0.15 Cox index, not significant), and an algebra posttest score 0.15 SD HIGHER online, also not significant. For English 9 it produced the largest negative result in either credit-recovery trial - a 23 percentage-point drop in credit recovery, effect size about -0.58 - which the authors trace substantially to online English teachers weighting tests and quizzes at 67% of the final grade against 40% in the comparison classes. Student-reported instructional experiences ran counter to the theory of action rather than in favour of it. Together with Heppen et al. (2017), the honest summary is that online credit recovery has never beaten a teacher-led class on any outcome in either randomised trial, and does clear harm when the model removes the teaching role from the room rather than merely putting the curriculum on a screen.

Minimal (randomised) - student-level random assignment within school-by-subject blocks with demonstrated baseline equivalence. The live threat is differential non-response on the study-administered test, not selection into treatment.

This IS the replication - it is the second and only other experimental test of online credit recovery, following Heppen et al. (2017) in Chicago, and the two disagree on algebra. Chicago found the online algebra posttest 0.19 SD LOWER and credit recovery 12 pp lower; Los Angeles found the algebra posttest 0.15 SD HIGHER (not significant) and credit recovery 6 pp lower (not significant). English 9 in Los Angeles, however, produced the largest negative in either study - a 23 pp drop in credit recovery, effect size about -0.58. The authors state explicitly that they cannot say whether the divergence is due to provider or instructional model. The reading this archive should carry is that the two trials differ on exactly the variable the cluster is about - how much of the teacher was removed from the room. Recorded as `mixed` because the pair of trials genuinely disagrees on the algebra outcome.