New Technology in Schools: Is There a Payoff?

591 LEA-year observations from the 150 Local Education Authorities in England, 1999-2003 (differenced panel). School-level first-stage regressions use the DfES ICT school survey. All second-stage achievement results are PRIMARY schools only - the instrument has no power for secondary schools.

English primary-school pupils taking the Key Stage 2 national tests at age 11 in English, Mathematics and Science, 1999-2003, aggregated to Local Education Authority level.

THE POLICY SHOCK: government ICT grants in England were distributed to Local Education Authorities. In 1999 and 2000 allocation ran through a BIDDING process, so money followed the LEAs that wrote the most enthusiastic proposals - i.e. it was endogenous to how much the authority cared about ICT. From 2001 the rule changed to a FORMULA based on school numbers, pupil numbers and a population-density (sparsity) adjustment, explicitly to make the system more equitable. Because the two bases were unrelated, the switch mechanically created large winners and losers with no reference to anything about pupils. The scale of the surrounding investment is important context: between 1998 and 2002 English primary-school ICT spending rose over 300 percent, from about GBP 3,600 to GBP 12,900 per school (roughly 2 percent of total expenditure), and secondary spending nearly doubled from GBP 40,100 to GBP 75,300 (about 3 percent). IDENTIFICATION, precisely: the instrument is Index_i x Policy-on, where Index_i is the FITTED log share of national ICT funding an LEA would have received under the post-2001 formula MINUS its ACTUAL log share in 1999/2000 - i.e. the size of the windfall or shortfall the rule change handed it - interacted with a post-2001 indicator. The first-stage regresses the change in log ICT funding per pupil on this, with LEA fixed effects (or Index_i as a linear trend control), year dummies, and controls for the formula inputs themselves (sparsity, pupil numbers, school numbers) plus the pupil-teacher ratio, so the instrument is not simply picking up the formula variables. Standard errors are clustered at LEA level; regressions are weighted by LEA pupil numbers. FIRST STAGE STRENGTH AND WHERE IT EXISTS: highly significant at LEA level with F statistics well above the Staiger-Stock weak-instrument critical values - a 10 percent increase in an LEA's relative share of ICT funding produced about a 9 percentage point increase in ICT funding per pupil. At SCHOOL level, using the computer-pupil ratio from the DfES ICT survey, the instrument is positive and significant for PRIMARY schools (a 10 percent share increase raises the computer-pupil ratio by about 0.11 percentage points) but exactly ZERO for secondary schools - so the authors restrict all outcome analysis to primary schools, which is the honest move and also halves the study's reach. THE VALIDITY CHECKS ARE GOOD: no significant relationship between the instrument and PRE-policy funding trends; winner and loser LEAs track each other identically in the years before 2001; no relationship between the instrument and 1999 LEA characteristics such as unemployment claims; the pre-change funding shares in 1999 and 2000 are very highly correlated, so LEAs did not pre-empt the rule change in their bidding; and Bertrand-Duflo-Mullainathan serial-correlation correction (collapsing to one pre- and one post-observation per LEA) leaves the English result significant at 5 percent (0.023) and Science marginal at 10 percent (0.019). NO CROWDING OUT: the instrumented growth in ICT expenditure is statistically insignificantly and substantively unrelated to other LEA funding, so the estimates are the effect of EXTRA resources spent on ICT holding other inputs constant - not of a reallocation away from something else. WHAT THE COMPARISON GROUP GOT: less ICT money and otherwise business as usual. This is an INPUT-EXPENDITURE study, not a curriculum trial - nobody replaced a teacher with software; the counterfactual LEAs simply grew their ICT spending more slowly. THE ESTIMAND IS NARROWER THAN IT LOOKS, and the authors are explicit: this is an Average Causal Response, weighted toward LEAs whose ICT funding growth ran at 50-80 percent per year. Big winners (top quartile of Index) averaged about 50 percent ANNUAL growth in ICT funds after 2001; big losers about 20 percent. So the identified effect is of enormous funding increases in authorities already functioning well, not of marginal spending anywhere. CAVEAT THIS ARCHIVE ADDS, which the paper does not discuss: the outcome is the PROPORTION OF PUPILS REACHING "LEVEL 4 OR ABOVE" at Key Stage 2 - a threshold measure that was simultaneously the English government's headline accountability target and the number published in School Performance Tables. Threshold measures are exactly where schools under pressure concentrate effort on borderline pupils, so some of the measured gain could be redistribution of teacher attention toward the level-3/4 boundary rather than a general increase in learning. The paper provides no continuous-score result to rule this out. A second internal tension worth noting: the ICT survey shows ICT is "substantially used" in teaching English by about 65 percent of primary schools, in Maths by about 56 percent, and is also important in Science - yet the effect is clear in English, borderline in Science and absent in Maths, so usage intensity alone does not order the results. GRADE B: a well-executed IV off an administrative rule change with a strong first stage, a clean pre-trend placebo, a serial-correlation correction and a crowding-out check - the methodology's "difference-in-differences with clean shocks / credible quasi-experiment" row. Not A because there is no randomisation, the second stage rests on 150 aggregated LEAs, the outcome is a threshold pass rate, the Science result is only borderline, and the estimate is local to authorities that received very large funding shocks. VERSION READ: full text of IZA Discussion Paper 2234 (July 2006), the openly available version of the article published in The Economic Journal 117(522), 1145-1167 (2007); abstract, tables and conclusions were read in full, and the typeset EJ version was not separately re-checked.

The strongest pro-technology natural experiment in this cluster, and it is narrower than its headline. Instrumenting ICT expenditure with the 2001 English funding-formula change, a doubling of ICT funding per pupil raises the proportion of 11-year-olds reaching Level 4 or above by about 2 percent in ENGLISH (IV coefficient 0.020, SE 0.007, and 0.022, SE 0.008 with controls; both significant at 1 percent) and by about 1.6 percent in SCIENCE (0.016, SE 0.009; borderline at the 10 percent level), and does nothing at all in MATHEMATICS (0.002-0.007, SEs 0.006, insignificant, point estimates near zero). The English effect is notable against a backdrop where average pupil scores in that subject grew about 7 percent over 1999-2003. Two features make this an honest steelman rather than a slam dunk. First, OLS finds NOTHING (English +0.004, SE 0.004; Maths -0.006) - the positive result exists only under instrumentation, which is the opposite of the usual pattern where naive correlations overstate ed-tech and causal designs deflate them, and it means the whole result rests on the instrument being valid. Second, the estimand is an Average Causal Response weighted toward authorities whose ICT budgets were growing 50-80 percent a year, in LEAs that had lower overall per-pupil spending but BETTER existing standards, on top of infrastructure installed since the mid-1990s, with the new money going largely into updating resources and teacher skills. The authors' own conclusion is conditional and should be quoted as such: it was "the joint effect of large increases in ICT funding coupled with a fertile background for making an efficient use of it that led to positive effects". Timing evidence supports a slow build - the effect in English is 0.013 (SE 0.005) in the immediate year after the change, rising to 0.026 two periods later and 0.031 three periods later - so this is one of the few studies in the cluster where "it needed time" is actually supported by data rather than asserted.

Low. The identifying variation is a national funding-formula change interacted with each authority's pre-existing share, with LEA fixed effects, verified absence of differential pre-trends, and no relationship between the instrument and 1999 LEA characteristics - pupil composition does not move with the instrument. The live threats here are instrument validity and the threshold outcome measure, not heredity.

This is the outlier positive in a literature of nulls, and the authors say so outright: with the exception of Banerjee et al. (2004) on Indian urban slums, every prior economic study - Angrist & Lavy (2002), Leuven et al. (2004/2007), Goolsbee & Guryan - found no positive causal relationship. The English ICT-funding-rule instrument has not been re-estimated by another team, so the specific result is unreplicated; the wider question has been tested many times with a split record, which is why `mixed`. The authors offer a substantive reconciliation rather than a dismissal: E-Rate money went to the most disadvantaged US areas where teachers were "novice or completely inexperienced with computers", whereas the English money went disproportionately to LEAs with LOWER overall spending but BETTER educational standards, on top of infrastructure already installed since the mid-1990s, and was spent largely on updating resources and TEACHER SKILLS rather than first-time hardware. Their claim is not "ICT works" but "large ICT increases work where the capacity to use them already exists".