Instruction time, Classroom Quality, and Academic Achievement

47,580 school-by-grade-by-subject cells covering 16,154 schools in 72 countries, aggregated from 304,070 students

15-year-olds (mostly 9th and 10th graders) sitting PISA 2009

The Economic Journal 125(588), F425-F448; full text read from NBER working paper 19464 (September 2013). School-by-grade and school-by-subject fixed effects on PISA 2009, so the effect of instruction time is identified within a school and, in the tightest specification, within a school and subject. The authors are candid that this costs them almost all the variation: "less than three percent of the variation in the school-average instruction-time difference remains" in the fully saturated model, which is "inadequate variation... to generate a precise, unbiased estimate". The paper's real contribution is the interaction, not the main effect: it tests whether the return to time depends on classroom environment (student-reported disruption, noise, lost time at lesson start) and on instructional quality separately. Classroom-environment measures are survey items and are noisy.

Two findings, and the second is the important one. FIRST, the main effect deflates sharply with fixed effects: 0.068 (SE 0.008) PISA points per weekly minute with no fixed effects, 0.025 (0.006) with school-by-grade fixed effects, and 0.017 (0.008) with school-by-subject fixed effects — roughly half Lavy's estimate on the same scale. In their own translation, "the addition of one class per week would raise achievement by roughly 2 points... the addition of 50 minutes per week would raise achievement by roughly 1.25 points." SECOND, and this is what makes the source load-bearing, the return to time is conditional on the classroom being orderly: "an additional hour of weekly instruction time raises achievement by more than twice as much at the 75th percentile (0.025 standard deviations) than at the 25th percentile (0.011 standard deviations)" of classroom quality, and "schools with behaviors that place them in the lower tail in terms of classroom environment therefore are likely to realize LITTLE OR NO BENEFIT from increases in instructional time." Notably it is the classroom ENVIRONMENT — disruption — that moderates, not measured instructional quality, whose interactions are small and insignificant. Returns also diminish: the quadratic turns negative above roughly 500 minutes per week, above the 95th percentile of the distribution.

Low within the design. School-by-subject fixed effects absorb school composition, which is where family and child selection operates, and the comparison is between subjects inside the same school. The heterogeneity result is a different matter — classroom environment is not randomly assigned, and schools with orderly classrooms differ from schools without them in ways that include their intakes, so "time works better in orderly classrooms" is an association between two school characteristics rather than an identified interaction.

Directionally replicated by Bietenbeck and Collins, who reach 0.014-0.037 SD per weekly hour across twelve assessment waves using Lavy's specification — the same deflation from a different route. The conditional-return finding has no direct replication, but it rhymes with the extended-time evaluation record, where large dose increases in low-performing schools (Massachusetts ELT, Florida's extra literacy hour) return little or nothing.