The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review

39 studies reviewed narratively; the 13 most recent meta-analysed

US school children, studies published before 1996

Review of Educational Research 66(3), 227-268. THE SOURCE OF EVERY SUMMER-LOSS CLAIM IN CIRCULATION. Read at ABSTRACT depth for the article itself, supplemented by the first author's own 2003 ERIC Digest summary read in full - a real limitation, and the reason the widely quoted 'one tenth of a standard deviation' figure is NOT recorded below: it could not be verified in primary text. Grade D is a design judgement about what is pooled: the constituent studies are seasonal comparisons on fixed-form paper tests with age-spreading score scales, and von Hippel and Hamrock later show that 'the entire summer learning literature reviewed in a 1996 meta-analysis was potentially vulnerable' to two specific measurement artifacts - scaling, and the change of test form between spring and autumn. That does not make the review incompetent; it makes its inputs uninterpretable at the precision claimed.

The package that entered policy: 'achievement-test scores decline over summer vacation' and 'the summer loss equals about one month on a grade-level equivalent scale'. From Cooper's own later summary: 'summer learning loss equaled at least one month of instruction as measured by grade level equivalents... summer loss was more pronounced for math facts and spelling than for other tested skill areas... more pronounced for math overall than for reading overall.' THE SOCIO-ECONOMIC CLAIM IS NARROWER THAN ITS REPUTATION, and the narrowing matters: 'all students, regardless of the resources in their home, lost roughly equal amounts of math skills over summer. However, substantial economic differences were found for reading. On some measures, middle-class children showed GAINS in reading achievement over summer, but disadvantaged children showed losses. Reading comprehension scores of both income groups declined, but the scores of disadvantaged students declined more.' Neither gender, ethnicity nor IQ moderated the loss. Note that the socio-economic finding is READING ONLY and rests partly on grade-equivalent scales, which is precisely the metric the later critique attacks.

High for the socio-economic component, which is the part most used politically. Summer environments differ by family in resources, books, travel and supervision - and the families supplying those also supply alleles. Nothing in a seasonal comparison distinguishes them, and the review makes no attempt to. The mean loss across all children is a different and much less confounded quantity, since it does not require a between-family comparison.

Partially replicated, and it is worth being precise about which half. THE MEAN LOSS SURVIVES: on adaptive IRT-scaled tests immune to the form-change artifact, Atteberry and McEachin find negative mean summer change at every grade from 1 to 8 in both subjects across roughly 18 million students, and von Hippel himself concedes 'summer learning is slow for nearly all children, including children from advantaged groups'. THE MATHS-WORSE-THAN-READING ASYMMETRY ALSO SURVIVES. THE GAP CLAIM DOES NOT: von Hippel and Hamrock find the evidence 'inconsistent regarding whether gaps grow faster during school or during summer'; Kuhfeld, Condron and Downey find on 2.5 million children that the Black-White gap WIDENS during school and NARROWS over summer because White students lose more; Atteberry and McEachin find race and socio-economic status together explain only about 4% of the variance in summer loss; and Quinn and colleagues find much less summer loss in the ECLS-K:2011 cohort than in ECLS-K:1998.