The Evidence on Teaching

Laptop multitasking hinders classroom learning for both users and nearby peers

Sana, F., Weston, T., & Cepeda, N. J. · 2013

grade Crctindependentmixednumbers spot-checked
Sample
Experiment 1 n = 40 (20 multitasking, 20 not); Experiment 2 n = 38 (19 in view of multitasking confederates, 19 not). 4 sessions of up to 20 seats each per experiment.
Population
York University undergraduates in an Introductory Psychology course, mean age 18.9 (SD 2.0), participating for course credit; all had to own a laptop. A simulated classroom, not a real course.
Design
Two randomized laboratory experiments in a mock lecture theatre. In Experiment 1 all participants took notes on their own laptops during a university-style lecture; half, randomly assigned, were also given 12 unrelated online tasks to complete "at any point" during the lecture. In Experiment 2 a fresh cohort took notes with PAPER AND PENCIL while confederates seated in front of them multitasked on laptops; participants were randomly given seat numbers as they entered, and instruction sheets designating condition were pre-placed at seats, so allocation to "in view" versus "not in view" was random from the participant's side. Fidelity control is unusually good for this literature: an experimenter observed every participant at 3-4 minute intervals, probed anyone off-task, and discarded data from anyone probed more than twice. Notes were scored blind for quality on a 1-5 rubric. WHAT THIS IS AND IS NOT: it is a well-run efficacy experiment on undergraduates in a single simulated lecture, with a researcher-designed comprehension test taken minutes later and no follow-up. It is NOT a school trial, the samples are tiny (n = 40 and 38, unregistered, no power analysis reported), the effect sizes are correspondingly enormous and should be read with the small-study discount METHODOLOGY.md requires, and there is no evidence here about durability. One contamination the authors disclose and that biases Experiment 1 DOWNWARD: the non-multitaskers in Experiment 1 were sometimes themselves in view of multitaskers. Note also that the authors' own recommendation is explicitly against banning laptops — they propose etiquette rules and seating conventions instead — which is not how the paper is usually cited. Independent academic work (McMaster, York); no funder named.
Key findings
Students who browsed the web during a lecture scored 11 percentage points lower on the comprehension test than students who did not (0.55 vs 0.66; F(1,38) = 10.2, p = .003, omega-squared = .20; d about 0.95 computed from the reported means and SDs). The finding that matters for a school's device policy is the second one: students taking notes on PAPER who merely sat in view of two multitasking peers scored 17 percentage points lower than students with a clear view (0.56 vs 0.73; F(1,36) = 21.5, p < .001, omega-squared = .36; d about 1.42) — an effect the authors note is nearly twice the self-distraction effect. That converts device use from a private choice into a NEGATIVE EXTERNALITY: the cost falls partly on classmates who made the opposite choice, which is the argument that a laptop-or-phone policy is a coordination problem rather than a matter of individual responsibility. Mechanism evidence points different ways in the two experiments: multitaskers' own notes were significantly worse (2.7 vs 4.1 on a 5-point rubric), but in-view participants' notes were NOT worse, so the peer effect is not running through note-taking. The caveat that most changes how it should be read: these are two 40-person lab sessions with a homemade quiz taken 30 minutes later, and effect sizes near d = 1 from samples that small are exactly the pattern this archive discounts.
Genetic confound
Minimal (randomized allocation to condition in both experiments).
Replication notes
The two halves of this paper have very different replication records and should not be cited as one finding. Experiment 1 — that a student who browses during a lecture learns less — was already well-established when this was published (the authors say so, citing Hembrooke & Gay 2003, Kraushaar & Novak 2010, Wood et al. 2012) and has been reproduced many times since; it is about as safe as anything in this literature. Experiment 2 — the NEARBY-PEER externality — is the novel claim, is what made the paper famous, and rests on a single 38-person laboratory session. We have not located an independent direct replication of the peer effect at anything like the same level of control, and it has not been demonstrated in a real classroom or with an external achievement measure. Treat the self-distraction result as replicated and the externality result as essentially unreplicated; the combined flag is `mixed`.
DOI / URL
10.1016/j.compedu.2012.10.003

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Lecture comprehension test, laptop multitasker vs non-multitasker (Experiment 1)percentage points / d0.55 (SD 0.11) vs 0.66 (SD 0.12), i.e. 11 percentage points lower; F(1,38) = 10.2, p = .003, omega-squared = .20; d about 0.95 computed from reported means/SDs. Interaction with question type (simple vs complex) not significant.researcher-designedabout 30 minutes after a single simulated lectureactive-alternativeend-of-treatmentdomain-skill
Lecture comprehension test, in direct view of a multitasking peer vs not (Experiment 2)percentage points / d0.56 (SD 0.12) vs 0.73 (SD 0.12), i.e. 17 percentage points lower; F(1,36) = 21.5, p < .001, omega-squared = .36; d about 1.42 computed from reported means/SDs. Participants in BOTH conditions took notes on paper — the only difference is what was on the screen in front of them.researcher-designedabout 30 minutes after a single simulated lectureactive-alternativeend-of-treatmentdomain-skill
Quality of the notes the laptop displaced (Experiment 1 mechanism / displacement)1-5 blind-scored rubricmultitaskers 2.7 (SD 1.2) vs non-multitaskers 4.1 (SD 1.0); t(34) = 3.6, p = .001, omega-squared = .23 — the browsing displaced note-taking effort, and multitaskers completed 9 of the 12 online tasks, so the secondary task genuinely consumed attentionresearcher-designedduring the lectureactive-alternativeend-of-treatmentbehaviour
Note quality of students in view of a multitasker (Experiment 2 mechanism)1-5 blind-scored rubricNOT reduced — in-view participants took notes of comparable quality to the clear-view group, so the peer externality is not mediated by worse note-taking; it looks like attentional captureresearcher-designedduring the lectureactive-alternativeend-of-treatmentbehaviour

Cited by