The Evidence on Teaching

Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries

Lavy V · 2015

grade Bquasi-experimentindependentmixed
Sample
137,083 pupil-subject observations in the 22-country OECD sample; parallel samples of 14 Eastern European and a set of developing countries, from PISA 2006 across more than 50 countries
Population
15-year-olds sitting PISA 2006 in maths, science and language
Design
The Economic Journal 125(588), F397-F424. Full text read from the September 2012 revision of NBER working paper 16227, so numbers may differ marginally from the published version. Identification is WITHIN-PUPIL ACROSS-SUBJECT: each student contributes three observations (maths, science, language) and pupil fixed effects absorb everything about the child and the family, so the effect is identified off a school giving more hours in one subject than another. That design kills the usual confound — able children sorting into high-hours schools — and is why this grades B. Three real weaknesses. (1) Instructional time is SELF-REPORTED by students in banded categories ("less than 2 hours", "2 to 4", "4 to 6", "more than 6"), with mid-values imputed; measurement error here is not classical. (2) Within-pupil variation cannot rule out subject-specific sorting or a school choosing to spend more hours on the subject its pupils are worse at, which biases toward zero, or on the subject it is best at teaching, which biases away. (3) The headline effect size is scaled to the WITHIN-pupil standard deviation (38.8 PISA points), which is less than half the between-pupil SD (84.4) — the same coefficient expressed on the conventional between-pupil scale is 0.07 SD, not 0.15. Both numbers are in the paper; only the larger one gets quoted.
Key findings
One additional weekly hour of classroom instruction in a subject raises that subject's PISA score by 5.76 points — 0.15 SD of the within-pupil distribution, and 0.07 SD of the between-pupil distribution. Lavy is explicit that this is far below the naive OLS estimate, which "overstate[s] the extent to which countries like the United States might catch up to other developed nations by increasing instructional time." The dose-response is CONCAVE and the concavity is the actionable finding: the marginal hour in the 2-3 hour range is worth 4.20 points, but in the 4+ hour range only 2.48 points, against an average of 5.76 — "the first two hours of instruction have the highest effect." Effects are larger for girls, immigrants and low-SES pupils. The developed/developing split is stark: the OECD and former-Soviet Eastern European estimates are similar to each other, but in developing countries an extra hour is worth only 2.99 points, 0.06 of the within-pupil SD and 0.04 of the between-pupil SD — "about half the effect size estimated for the OECD developed economies". (The paper's introduction states 0.025 SD for this same estimate; the results section figure of 0.06 is the one attached to the reported coefficient, and the two are inconsistent in the working-paper text.) The equity direction REVERSES between samples, which is rarely reported: in the OECD the effect is larger for low-SES pupils, but in the developing-country sample it is 26 PERCENT LOWER for pupils from low-education families. Instructional time is more productive where schools face accountability measures or hold autonomy over budgets and over hiring and firing teachers, which reframes hours as an input whose return depends on the quality of what fills them.
Genetic confound
Low, and by construction. Pupil fixed effects difference out every stable child and family trait including ability and parental education, which is precisely the channel that contaminates cross-school comparisons of instructional time. What survives is subject-specific confounding — anything that makes a school allocate hours differently across subjects in a way correlated with pupils' subject-specific aptitude — which is a much smaller worry than the one the design removes.
Replication notes
The sign replicates and the magnitude does not, which is why this is mixed. Bietenbeck and Collins re-ran Lavy's own specification on six PISA waves (2000-2018) and six TIMSS waves (1995-2015) in the same 22 OECD countries: they reproduce PISA 2006 exactly at 0.058 SD on the 100-point international scale, and then get 0.014 to 0.031 SD in every other PISA wave and 0.015 to 0.037 SD in every TIMSS wave, attributing the gap to PISA 2006 having measured instruction time CATEGORICALLY where later waves used a continuous lessons-times-minutes construction. Rivkin and Schiman, using school-by-subject fixed effects on PISA 2009, land independently at 0.017-0.025 PISA points per weekly minute, roughly half Lavy's. Lavy himself reports that his estimates are "almost identical" to Israeli results from two other identification strategies. The defensible central estimate for an extra weekly subject-hour on an international assessment is therefore about 0.02-0.03 SD on the conventional scale, not the 0.058 or 0.15 figures that get quoted.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Subject test score per additional weekly hour of instruction, OECD samplePISA points and SD+5.76 PISA points = 0.15 SD of the within-pupil distribution = 0.07 SD of the between-pupil distributionstandardizedsame-year PISA 2006 assessment at age 15business-as-usualend-of-treatmentdomain-skill
Concavity of the dose-responsePISA points per marginal hour4.20 points per hour in the 2-3 hour range versus 2.48 points per hour in the 4+ hour range, against a 5.76 average — the first two hours carry most of the returnstandardizedsame-year PISAbusiness-as-usualend-of-treatmentdomain-skill
Subject test score per additional weekly hour, developing-country samplePISA points and SD+2.99 points = 0.06 of the within-pupil SD and 0.04 of the between-pupil SD, "about half the effect size estimated for the OECD developed economies"; Eastern European estimates resemble the OECD onesstandardizedsame-year PISAbusiness-as-usualend-of-treatmentdomain-skill
Heterogeneity of the instruction-time effectdirection of moderationOECD sample — larger for girls, for immigrants and for pupils from low socio-economic status families. Developing-country sample — larger for girls (+38%) and first-generation immigrants (+67%) but 26% LOWER for pupils from low-education families, i.e. the equity direction reverses. Larger overall in countries with school accountability measures or school autonomy over budgets and over hiring and firing teachersstandardizedsame-year PISAbusiness-as-usualend-of-treatmentdomain-skill

Cited by