Value added or misattributed? A multi-institution study on the educational benefit of labs for reinforcing physics content

9 lab courses at 3 institutions; N = 2,617 students on the final-exam analysis (df = 2,615)

US undergraduate introductory physics (calculus-based and algebra-based), three institutions spanning a Hispanic-serving public research university, a large public research university, and an elite private university.

Not randomised — labs were OPTIONAL, and lab-takers were systematically stronger. The identification trick is what makes it credible: each exam question was classified as lab-related (a lab activity covered everything needed to answer it) or non-lab-related, and each student's DIFFERENCE score (fraction correct on lab-related minus fraction correct on non-lab-related items) was computed. Comparing mean difference scores between lab-enrolled and non-lab-enrolled students differences out student ability and item difficulty simultaneously. This is a within-student difference-in-differences and it is a genuinely strong design for an observational setting. Outcomes are the instructors' own course exams — i.e. the measure is if anything biased TOWARD finding a lab effect, since it is the aligned, high-stakes assessment the students were actually working for. Three pre-registered-style questions were asked in advance: overall exams, conceptual items only, and midterms (to catch effects that fade by the final).

OFF-SCOPE FOR THIS ARCHIVE (undergraduate) BUT RECORDED IN FULL because it is the single most damaging study in the entire laboratory literature and a founder will be handed it. Across nine introductory physics lab courses at three very different institutions — 2,617 students — the measured value added by taking the lab, on the exam content the lab was explicitly designed to reinforce, was +0.24 ± 0.67 percentage points. Not 'small and positive'. Zero, measured precisely enough that anything above about 1.5 percentage points is excluded. Conceptual items only: -0.62 ± 1.58. Midterms, to catch fadeout: also null. The authors' phrase is 'universally and precisely no added value'. The measure was the instructors' own aligned course exams, so this is a null on the most favourable available outcome. WHAT IT DOES AND DOES NOT SHOW: it kills the specific claim that labs reinforce content learning from lectures. It says nothing about labs whose goal is experimental skill, measurement, apparatus fluency or scientific practices — the authors explicitly recommend redirecting labs to those goals. And it is 18-to-22-year-olds, not schoolchildren.

Medium in principle (self-selection into the optional lab) but the difference-score design removes stable between-student ability differences, which is the main channel. Residual risk is an interaction between ability and lab-related content, which the authors argue against.

Directly extends and replicates the authors' own earlier two-course, single-institution study (Wieman & Holmes 2015, AJP), which found the same null to within 1% with 2% uncertainty. Nine courses, three institutions, three student populations, two exam types and two timepoints all agree. Within the undergraduate physics literature this is about as replicated as a null gets.