New evidence on the importance of instruction time for student achievement on international assessments

Six PISA waves 2000-2018 (21,859 to 164,600 students per wave) and six TIMSS waves 1995-2015, across the same 22 developed OECD countries Lavy studied

15-year-olds (PISA) and fourth- and eighth-graders (TIMSS) in 22 developed OECD countries

Journal of Applied Econometrics 38(3), 423-431; full text read from the author's working-paper version dated 25 August 2021. A direct replication-and-extension: the authors run Lavy's own within-student across-subject fixed-effects specification, unchanged, on every other wave of the two major international assessments. This is the design the archive most wants and almost never gets — the same estimator, the same countries, different data collections. Effects are expressed on the 100-point international scale, so 0.058 here is the same coefficient Lavy reports as 0.15 of the within-pupil SD; the three different denominators in circulation for one number are the reason this source exists.

"We successfully replicate the positive effect... IN A NARROW SENSE. Extending the analysis to other international assessments, we find effects that are CONSISTENTLY SMALLER IN MAGNITUDE." PISA 2006 — the wave Lavy used — reproduces exactly at 0.058 (SE 0.004) SD per weekly hour. Every other PISA wave is far lower: 0.019 (2000), 0.027 (2009), 0.031 (2012), 0.020 (2015), 0.014 (2018). Every TIMSS wave is likewise lower: 0.037 (1995), 0.037 (1999), 0.024 (2003), 0.015 (2007), 0.019 (2011), 0.017 (2015). The diagnosis is measurement, not sampling: PISA 2006 collected instruction time as a CATEGORICAL band while later waves constructed it continuously from lessons times minutes, and "PISA 2006 with its categorical measurement produces by far the largest point estimate." The authors' conclusion is that differences in instruction time are "less important than previously thought" for explaining international achievement gaps.

Low for the same reason as Lavy — student fixed effects difference out every stable child and family characteristic, and identification comes from a school giving more hours in one subject than another to the same child. The authors are explicit that "the non-experimental nature of the data does not allow us to completely rule out the influence of confounding unobservables", but the surviving confounds are subject-specific, not genetic.

This IS the replication, so the field is not-applicable. Its verdict converges with Rivkin and Schiman, who reach 0.017-0.025 PISA points per weekly minute on PISA 2009 using school-by-subject fixed effects — also roughly half Lavy's headline. Two independent teams, two different corrections, the same direction of deflation.