The effects of increased learning time on student academic and nonacademic outcomes: Findings from a meta-analytic review

more than 7,000 studies screened; 30 met What Works Clearinghouse-style evidence standards and entered the meta-analysis

US school students in increased-learning-time programmes, including summer schools and out-of-school-time programmes

REL Appalachia report REL 2014-015, 'What's Known' series, June 2014; ERIC ED545233. THE HIGHEST METHODOLOGICAL BAR APPLIED TO THIS QUESTION - studies had to meet WWC-style design standards to be pooled at all, which is why 30 of more than 7,000 survived. The report sets an a priori bar for 'educational significance' at 0.25 SD following Hill, Bloom, Black and Lipsey. Funded by IES, independent of programme developers.

'There is insufficient evidence to suggest that increased learning time is an effective approach for promoting the academic outcomes of all students in all settings', and specifically: 'this review did not find that summer schools had a statistically significant effect on student outcomes.' The numbers: summer-school literacy g = 0.16 (95% CI -0.04 to 0.36, p = 0.12, k = 6); out-of-school-time literacy g = -0.04 (-0.21 to 0.12, p = 0.61, k = 7); out-of-school-time mathematics g = 0.03 (-0.02 to 0.08, k = 5); academic motivation g = 0.04 (0.00 to 0.08, p = 0.04, k = 10) - the only statistically significant estimate in the table, and it is a quarter of the report's own significance bar; social-emotional skills g = 0.03 (-0.05 to 0.10, k = 11). THE EXPLANATION FOR THE GAP WITH COOPER'S SUMMER-SCHOOL META-ANALYSIS IS STATED DIRECTLY: 'the 41 studies in that meta-analysis included mostly studies with less rigorous study design (for example, the studies did not use comparable control groups).' The same point is made about Cooper's full-day-kindergarten meta-analysis. This is the archive's design-quality rule appearing inside the source literature.

Low for the pooled estimates, since the inclusion rule requires comparable control groups. The relevant confound in this area is selection into voluntary summer programmes - families who enrol differ - and screening for design quality is precisely what removes it, which is why the screened estimates are so much smaller than the unscreened ones.

Its central claim - that design quality predicts effect size in this literature - is independently reproduced by Lynch, An and Mancenido, who report that summer-programme studies without a control group average 0.30 SD while comparison-group estimates average 0.09 SD, and by the RAND randomised trial, whose one causal effect was 0.08 SD in mathematics and dissipated by the following spring.