Zooming to Class? Experimental Evidence on College Students' Online Learning during COVID-19

551 students (550 randomised; 551 appear in the analysis tables), 38 class sections, 12 instructors, Fall 2020. Post-course survey completed by a voluntary sub-sample with extra-credit incentive.

POST-SECONDARY. Undergraduates at the United States Military Academy at West Point, taking the required "Principles of Economics" core course (introductory micro, macro and personal finance, Mankiw 8th ed.). Highly selective and highly structured - median SAT maths 650, congressional nomination required, no tuition, mandatory attendance at every class period, and the Academy randomly assigns semester, class hour, instructor and roommates as a matter of routine. Outside the archive's 4-18 age band, and the setting is atypical even for higher education - a residential military academy where students cannot skip class, during a pandemic.

WHAT "ONLINE" MEANS HERE: LIVE SYNCHRONOUS remote attendance at the same scheduled class hour, with the same instructor - the "Zooming" of the title. It is not asynchronous recorded video and not self-paced software. Both online and in-person sections were offered at every class hour and students were randomised WITHIN hour, so time of day is held constant. Syllabus, homework, problem sets, graded events and exams were identical. Crucially, each of the 12 instructors taught half their sections online and half in person, so instructor fixed effects hold the teacher constant - the design compares the same teacher's online and in-person students. Preregistered in the AEA RCT Registry as AEARCTR-0006969. All grades standardised to mean 0, SD 1. Specifications add instructor, class-day (which also controls exam version) and time-of-day fixed effects plus demographic controls; the estimate is stable across all five columns (-0.236 to -0.215). INFERENCE: clustered standard errors came out SMALLER than robust ones - a small-cluster symptom - so the authors report robust and wild-bootstrap p-values alongside; the headline survives both (p = .011 and .013). LIMITS THE AUTHORS NAME: the estimate bundles "online delivery" with "instructors teaching worse online", and the experiment cannot separate them; they argue this is what the sector actually experienced. Both arms were operating under COVID conditions (50% classroom capacity, distancing, testing), so the in-person counterfactual is itself degraded - which if anything biases the contrast toward zero. HETEROGENEITY CAUTION: the widely-quoted "worse for at-risk students" claim rests on subsample point estimates, NOT on significant interactions - Online x CEER<25th percentile is -0.232 (SE 0.174) and Online x CEER<50th is -0.203 (SE 0.150), neither significant. Read the heterogeneity as suggestive. FUNDING/INDEPENDENCE: no vendor and no product - the authors are West Point faculty studying their own institution's course, which is a conflict of a different kind (institutional, not commercial) and worth noting; the views are disclaimed as not those of the Academy or the Army. READ NOTE: read in full as IZA Discussion Paper No. 14356 (May 2021); the published AER: Insights abstract reports the identical headline (-0.215 SD), so the preprint numbers below stand.

Randomly assigning students to attend the same course by live video instead of in the room lowered final course grades by 0.215 SD (robust p = .011, wild-bootstrap p = .013), about 1.5 points out of 100 or half a plus/minus grade. The effect is present on invigilated exams and on daily homework, not just on participation-type marks, so it is not a grading artefact - the only clearly null category is the collaborative problem sets, which students could do together regardless of section. Students in the online arm reported studying slightly MORE minutes per day (+2.4, not significant) for the lower grade, which rules out simple effort substitution as the mechanism and points at the survey findings instead - large drops in reported ability to concentrate (about three-quarters of an SD), connection to the instructor and connection to peers. This is the cleanest randomised estimate available of "same teacher, same materials, same hour, screen instead of room", and it is negative.

Minimal (randomised) - students were randomly assigned to modality within class hour at an institution that already randomises course semester, instructor and class hour.

No direct replication of this experiment exists. The direction replicates across designs - Bettinger et al. (2017) find -0.25 to -0.33 SD on course grades from asynchronous online sections at DeVry, and Figlio, Rush & Yin (2013) find live lectures ahead of recorded ones in a randomised trial - but the magnitudes and the statistical status differ - Figlio's overall contrast was not significant, and Bettinger's design and treatment were both different. Recorded `mixed` rather than `replicated` because the specific quantity here (about -0.2 SD from attending the SAME live class remotely) rests on this one trial, and because the wider COVID-era literature is heterogeneous and mostly non-causal.