The Evidence on Teaching

The Impact of Internet Subsidies in Public Schools

Goolsbee, A., & Guryan, J. · 2006

grade Bnatural-experimentindependentmixednumbers spot-checked
Sample
Every public school in California, 1996-97 through 2000-01 - 7,991 to 8,812 schools per year across about 1,055 districts, together roughly 13 percent of US public school enrolment. The preferred all-schools regression uses 31,243 school-year observations (24,061 for applier-only specifications, 3,684 for schools in never-applying districts, 4,340 for non-applying schools in applying districts).
Population
US K-12 public schools and their students in California, school years 1996-97 to 2000-01. Achievement results are split into primary schools (highest grade served is 9 or below) and secondary/high schools (serving at least one grade above 9).
Design
THE INTERVENTION: the federal E-Rate, enacted in 1998, which subsidised school spending on internet access, telecommunications services and internal wiring at 20 to 90 percent depending on the school's share of National School Lunch Program eligibles and its urban/rural status. California received almost $937 million between 1998 and 2000. FUNDING AND INDEPENDENCE: academic economists at the University of Chicago GSB and NBER; Goolsbee and Guryan acknowledge NSF (SES-0312749), the Alfred Sloan Foundation, the American Bar Foundation and the Centel/Reuss fund. Data from the California Department of Education, USAC E-Rate application records, NCES CCD and the 1990 Census, with the Urban Institute supplying some of it. No vendor of anything - `independence: independent`. THE IDENTIFICATION PROBLEM AND THE FIX, which is the clever part: the statutory discount is a step function of the school-lunch share, but the operative discount is a DISTRICT rate computed as the enrolment-weighted average of the individually-eligible rates of the schools a district chooses to put on its application - so districts have an incentive to load high-poverty schools onto applications and the realised subsidy is endogenous. The authors document exactly this: school-lunch shares were 0.51 at applying schools, 0.40 at non-applying schools inside applying districts, and 0.33 at schools in never-applying districts. THEIR SOLUTION is a simulated instrument - the subsidy rate the district WOULD have received had it been forced to include every one of its schools, computable for every district including non-appliers and driven purely by the mechanical interaction of the statutory step function with pre-existing enrolment and poverty composition. The estimating equation regresses the ONE-YEAR CHANGE in internet-connected classrooms per full-time-equivalent teacher on this simulated subsidy, with school fixed effects (which, on a differenced dependent variable, absorb school-specific TRENDS), year effects, and a linear-plus-quadratic control in the school-lunch share so the subsidy is not merely proxying poverty. Standard errors are corrected for within-district correlation throughout. They also present a genuine regression discontinuity at the five school-lunch cutoffs (1, 20, 35, 50, 75 percent) using schools within one percentage point: a first-stage jump of .028 (.006) in the district subsidy rate, a placebo reduced form of .003 (.038) in pre-E-Rate 1996-97 (no differential trend), and an E-Rate-period reduced form of .055 (.030), significant at 10 percent - but the implied Wald estimate of 1.933 (1.149) is "more than ten times larger" than the preferred estimates and, in the authors' own words, "all of the RD estimates are quite imprecise, owing to the amount of data we must discard". The RD is therefore a robustness check, not the headline. FALSIFICATION CHECKS: appliers with the actual subsidy give .173 (.068); schools in NEVER-applying districts give .076 (.086), insignificant, so the base result is not driven by unobserved district characteristics correlated with poverty. A spillover test finds non-appliers respond only when the within-district spread of school subsidy rates is large. A diminishing-returns test (subsidy x initial internet per teacher = -.046, SE .017) is significant but far too small to explain the result. California equalises per-pupil spending and its Board of Education treats E-Rate as a discount rather than revenue, so subsidies did not trigger offsetting state funding cuts; dropping Los Angeles Unified, which alone drove the $230 million 2000-01 spike, does not change the results. WHY THIS NULL IS INFORMATIVE - THE FIRST STAGE IS STRONG AND DOCUMENTED. Internet- connected classrooms per teacher across the state ran .17, .26, .38, .54, .66 over the five years (roughly 4.5, 7.7, 11.8, 17.7 and 21.7 classrooms per school), and the share of schools with any connected classroom went from .46 to .84. The regression implies that a school with average pre-E-Rate growth would have reached only 0.396 connected classrooms per teacher by 2000-01 without the subsidy; it actually reached 0.664, "some 68 percent higher". The programme bought a large, real, measurable increase in connectivity, concentrated exactly where the digital-divide rationale said it should be - the coefficient is .158 (.049) for primary schools versus .062 (.058) for secondary, .173 (.050) urban versus -.013 (.067) rural, and .226 (.093) interacted with percent Black and .122 (.044) with percent Hispanic versus .029 (.052) for percent White. So the achievement null cannot be dismissed as "the money never reached the classrooms". WHAT THE COMPARISON SCHOOLS DID WITH THE SAME TIME: nothing was substituted - this is pure ADDITIONAL infrastructure. Less-subsidised schools ran their normal timetable with fewer connected classrooms; more-subsidised schools ran their normal timetable with more. Whatever the internet displaced inside those classrooms is not observed, and the paper makes no claim about it. GRADE B: a large, clean policy shock with a well-constructed simulated instrument, a demonstrated first stage, a placebo test on the pre-period and multiple falsification checks - the "difference-in-differences with clean shocks" row. Not A because there is no randomisation, the achievement estimates are reduced-form effects of a SUBSIDY RATE rather than of internet access itself, and the standard errors, while informative, are wide enough that only effects above roughly 0.1 school-level SD can be excluded. VERSION READ: the April 2005 accepted manuscript ("A version of this paper is forthcoming in the Review of Economics and Statistics"), retrieved via the Internet Archive from Goolsbee's Chicago Booth faculty page; complete with all eight tables. Its abstract matches the published RePEc/REStat abstract word for word, including the 68 percent figure. The earlier NBER WP 9090 (2002) abstract quotes 66 percent as the INCREASE, which was revised to 68 percent for publication; 66 percent is separately the actual LEVEL of classrooms online in 2000-01, and the two numbers are routinely confused.
Key findings
The E-Rate did what it was designed to do on access and nothing detectable on learning. First stage: a ten-percentage-point higher subsidy raised the growth rate of internet access by 0.0136 classrooms per teacher per year (coefficient .136, SE .042), a first-dollar price elasticity of -1.1 falling to about -0.4 at the mean 63 percent subsidy; by 2000-01 California schools had about 68 percent more internet-connected classrooms per teacher than they would have had without the programme (0.664 versus a counterfactual 0.396), with the response concentrated in urban, primary, and heavily Black and Hispanic schools and essentially zero in rural ones. Achievement: nothing. On Stanford 9 mean scores the subsidy coefficients are +.001 (SE .064) for math across all schools, -.025 (.083) for reading and -.003 (.224) for science; for primary schools +.023 (.067) math and +.019 (.087) reading; results for language, spelling and social studies are "substantively the same". The authors' words: "None of the estimates reported in the table are statistically different from zero, and all are quite small in magnitude", and "while the standard errors are large, they are tight enough to rule out a substantial effect" - at the top of the 95 percent CI, a typical 63 percent subsidy would buy about 0.099 school-level SD, against the 0.22 student-level SD Krueger reports for the Tennessee STAR class- size experiment, and at far lower cost per unit of effect for STAR. Two-year-lagged effects (Table 8) are no better: "Many of the point estimates go down over time, not up", which kills the "schools were still learning to use it" defence over this horizon. Non-test outcomes - taking advanced classes, share of graduates entering the University of California system, dropout rate - also show no significant impact. Their conclusion: "Judged as a means of improving student performance, however, we fail to find strong evidence of success. Despite the noticeable impact on the expansion of the Internet, estimated effects on test scores in a variety of subjects are indistinguishable from zero."
Genetic confound
Low. The identifying variation is a statutory discount schedule interacted with pre-existing district composition, with school fixed effects on a differenced outcome (absorbing school-specific trends), an explicit flexible control for the poverty share, and a verified null placebo in the pre-programme year - so pupil composition is not driving the estimates. Genes are not the live threat here; district application behaviour is, and that is what the simulated instrument addresses.
Replication notes
The California E-Rate discontinuity has been revisited in later work on broadband subsidies (e.g. Hazlett, Schwall & Wallsten's E-Rate broadband studies) with broadly similar achievement nulls, but no one has re-estimated this exact design on this exact data. As a claim about large-scale public technology funding, the finding sits in a split literature: nulls in Dynarski et al. (2007), a negative in Angrist & Lavy (2002) and Leuven et al. (2007), a positive in Machin, McNally & Silva (2007). What makes this study distinctive and hard to explain away is that the FIRST STAGE worked - connectivity rose sharply and measurably - so the achievement null is not a story about a programme that never happened.
DOI / URL
10.1162/rest.88.2.336

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
FIRST STAGE - internet-connected classrooms per full-time-equivalent teacherclassrooms per teacher per yearcoefficient .136 (SE .042) on the simulated district subsidy rate - a ten-percentage-point higher subsidy raises annual growth in access by 0.0136 classrooms per teacher, i.e. 1.36 percentage points more classrooms connected per year. Implied first-dollar price elasticity -1.1, marginal elasticity about -0.4 at the mean 63 percent subsidy. Statewide levels rose from .17 to .66 classrooms per teacher (about 4.5 to 21.7 classrooms per school) between 1996-97 and 2000-01, and the share of schools with any connected classroom from .46 to .84.administrativeannual, 1996-97 to 2000-01business-as-usualend-of-treatmentbehaviour
FIRST STAGE - total programme effect on connectivity by the last year of the samplepercenta school with average pre-E-Rate growth would have reached 0.396 internet-connected classrooms per teacher by 2000-01 without the subsidy; it actually reached 0.664 - "some 68 percent higher". Response concentrated where intended: primary .158 (.049) vs secondary .062 (.058); urban .173 (.050) vs rural -.013 (.067); percent Black .226 (.093) and percent Hispanic .122 (.044) vs percent White .029 (.052).administrative2000-01, three years after the programme beganbusiness-as-usualover-2yrbehaviour
Mean mathematics score, Stanford 9 (change from year t to t+1)school-level SD per unit subsidy rateall schools +.001 (SE .064), 95% CI at a 63 percent subsidy [-.080, .082]; primary +.023 (SE .067) [-.069, .099]; secondary -.197 (SE .178) [-.348, .100]. All insignificant. The primary-school point estimate implies a 90 percent subsidy would raise math scores by about 2 percent of a cross-school SD.standardizedone-year change, annually across 1997-98 to 2000-01business-as-usualunder-1yrdomain-skill
Mean reading score, Stanford 9school-level SD per unit subsidy rateall schools -.025 (SE .083) [-.121, .089]; primary +.019 (SE .087) [-.098, .122]; secondary -.298 (SE .199) [-.438, .062]. All insignificant.standardizedone-year changebusiness-as-usualunder-1yrdomain-skill
Mean science score, Stanford 9school-level SD per unit subsidy rateall schools -.003 (SE .224) [-.284, .280]; secondary -.051 (SE .222) [-.312, .248]. No primary estimate reported. Insignificant and very imprecise. The authors add that results for language, spelling and social studies are "substantively the same".standardizedone-year changebusiness-as-usualunder-1yrdomain-skill
Percent of students scoring above the national 75th percentile (math, reading, science)percentage points per unit subsidy ratemath all schools +.95 (SE .83), primary +1.17 (1.00), secondary -.32 (1.06); reading +.11 (.84), +.52 (.98), -1.12 (.89); science +.29 (1.21) and +.10 (1.22). Point estimates are larger here than for means and larger for primary than secondary - consistent with primary schools' larger investment response - but none is statistically significant and the primary/secondary difference is not significant either.standardizedone-year changebusiness-as-usualunder-1yrdomain-skill
Percent of students scoring above the national 25th percentile (the bottom-of-distribution measure) - every point estimate is NEGATIVEpercentage points per unit subsidy ratemath all schools -1.84 (SE .94) [-2.34, 0.03], primary -1.71 (.98), secondary -3.38 (3.14); reading -1.40 (1.62), primary -.34 (2.09), SECONDARY -5.31 (2.35) with a 63-percent-subsidy CI of [-6.30, -.39] that EXCLUDES ZERO; science -1.82 (2.64) and -1.79 (2.67). Recorded because the paper's blanket statement that "none of the estimates reported in the table are statistically different from zero" is not quite right for the secondary-reading cell - though with 24 estimates in the table one nominal hit is what chance delivers, and the authors do not build on it.standardizedone-year changebusiness-as-usualunder-1yrdomain-skill
Achievement two years after (t to t+2) - the "schools need time to learn to use it" testschool-level SD / percentage points per unit subsidy rateno improvement with time. Mean math all schools -.178 (SE .096), primary -.058 (.073), secondary -.042 (.393); mean science all schools +.573 (.476). The authors' summary: "Many of the point estimates go down over time, not up", and they explicitly reject the view that schools were learning to use the technology in a way that shows up on test scores over this horizon.standardizedtwo-year changebusiness-as-usual1-2yrdomain-skill
Non-test outcomes - advanced-class taking, share of graduates entering the University of California system, dropout ratenone detected"we found no significant impact of Internet connections on performance" on any of these. The authors caution it is "probably a stretch" to expect technology subsidies to show up here, especially since the connectivity first stage for high schools is itself weak.administrativeannual, over the sample periodbusiness-as-usualunder-1yrattainment

Cited by