The Impact of Summer Programs on Student Mathematics Achievement: A Meta-Analysis

37 experimental and quasi-experimental studies yielding 149 effect sizes (112 mathematics achievement, of which 96 from standardized tests across 34 studies; 16 attainment; 37 non-cognitive)

Pre-kindergarten through grade 12 students in summer programmes with a mathematics component

Review of Educational Research 93(2), 275-315; full text read from EdWorkingPaper 21-379 (July 2022 version). Hedges g with robust variance estimation. Inclusion required statistical adjustment for baseline equivalence wherever standardized baseline differences ran between 0.05 and 0.25 SD, so the corpus is design-screened. The paper supplies the comparison that makes it valuable to this archive: it reports what weaker designs in the same literature produce.

Summer programmes move mathematics by 'an average weighted impact estimate of +0.10 standard deviations'. The estimate on standardized maths tests specifically is also +0.10 SD; broader attainment outcomes +0.11 SD; non-cognitive outcomes +0.11 SD. The distribution is not uniformly favourable: 'of the 112 effect sizes... 72 were positive in sign (64%), and 29 of these were statistically significant. Thirty-seven effect sizes were negative in sign (33%).' Effects are similar in higher- and lower-poverty settings, and programmes devoting more hours per day to mathematics still produce +0.10 SD - more time inside the programme does not buy more. THE DESIGN GRADIENT IS THE MOST USEFUL NUMBER IN THE PAPER: prior syntheses' comparison-group estimates ran at 0.09 SD while pretest-posttest-only studies with no control group averaged 0.30 SD - a threefold inflation attributable to design alone, and the same mechanism Kidron and Lindsay identify in Cooper's summer-school meta-analysis.

Low to medium. Baseline-equivalence screening removes the crudest selection, but summer programmes are largely voluntary and the families who enrol and attend differ from those who do not in ways correlated with the outcome. The RAND randomised trial is the cleanest evidence on this exact point: its intent-to-treat effect was 0.08 SD in the first year while the correlational high-attender estimates ran 0.11 to 0.16 SD, and the authors were explicit that the latter 'are correlational (exploratory) rather than causal'.

Converges with the WWC-screened REL meta-analysis, which finds summer-school literacy g = 0.16 and not significant, and with the RAND National Summer Learning Project's randomised 0.08 SD in mathematics that dissipated by the following spring. Three independent syntheses and one RCT land in the 0.08-0.16 SD band; the outlier estimates in the literature are all from designs without control groups.