Head Start Impact Study: Final Report
Puma, M., Bell, S., Cook, R., Heid, C., et al. (US DHHS/ACF) · 2010
grade Arctindependentunreplicatednumbers spot-checked
Sample
4,667 randomized (2,559 age-3 cohort; 2,108 age-4 cohort)
Population
Head Start-eligible children nationally.
Design
The only large national RCT of Head Start — congressionally mandated, designed on the recommendation of the independent Advisory Committee on Head Start Research and Evaluation, sponsored by ACF (the agency that administers Head Start; the ACF project officer is on the author line) and executed under contract by Westat with Abt, AIR, Urban Institute and others. Two independently randomised age cohorts (entering at 3 and at 4), each followed to the end of 1st grade; the 3-year-old cohort has an extra measurement wave (end of the age-4 year), during which many treatment children were in a SECOND Head Start year. No IQ test was administered — PPVT (receptive vocabulary) is the most g-loaded proxy available. Roughly 10,000 impact comparisons were tested across two cohorts, four waves and many outcomes, so the report grades evidence in three tiers: STRONG (p<0.05 and survives Benjamini-Hochberg), MODERATE (p<0.05 only), SUGGESTIVE (p<0.10, may or may not survive). Nearly every effect size in the summary exhibits is reported at the p<=0.10 threshold, so several "impacts" are suggestive-only. No-shows ~15-20% and crossovers ~14-17%; both ITT and IV-adjusted participation impacts are reported, and everything below is ITT.
Key findings
The definitive test of Head Start as actually delivered, and the cohort distinction is load-bearing. 4-YEAR-OLD COHORT: six small language/literacy gains at the end of the Head Start year (0.09-0.25 on direct assessments, 0.31 on parent report), never any math or pre-writing or promotion impact in any year, exactly zero cognitive impacts at the end of kindergarten, and at 1st grade only a suggestive PPVT +0.09; social-emotional zero in the Head Start year and kindergarten, then mixed and partly unfavourable at 1st grade. 3-YEAR-OLD COHORT: a broader end-of-year effect including math (Applied Problems +0.15) and PPVT +0.18, of which only two measures survive to the age-4 year and none survive to 1st grade except a suggestive Oral Comprehension +0.08; it is also the only cohort with social-emotional impacts, all of them parent-reported, and it carries the study's single unfavourable cognitive impact (teacher-rated math ability -0.19 at kindergarten). Across both cohorts, the durable findings are health-access and parenting, not learning: dental care +15pp (4yo) and +17pp (3yo) at the end of the programme year, gone by 1st grade, and health insurance coverage +4pp still present for the 4-year-old cohort at 1st grade. The counterfactual is an active alternative, not nothing, and both groups remain far below national norms throughout.
Counterfactual
NOT nothing, and the report says so explicitly: Head Start was tested "against a mixture of alternative care settings rather than a condition of no services". 4-year-old controls (Exhibit 1, spring 2003): 39.7% parent care only, 35.3% other centre-based care, 13.8% crossed into Head Start anyway, 11.2% other care. 3-year-old controls: 38.4% parent care, 25.2% other centre-based, 17.8% Head Start crossover. Controls who were in non-parental care were in it 4-5 MORE hours per week than the Head Start group. In the 3-year-old cohort's second year the control group was given access to Head Start, so the age-4 contrast is badly contaminated: ~50% of controls in centre-based care were in Head Start by spring 2004.
Genetic confound
RCT. The effect ordering is diagnostic: largest on parent-report, smallest on the most g-loaded measure.
Replication notes
No second national RCT of Head Start exists, so `unreplicated` refers to the design, not to the fadeout pattern, which is one of the most replicated regularities in the literature. The significant end-of-Head-Start-year cognitive effect sizes in this study run 0.09-0.31 (4yo cohort) and 0.10-0.35 (3yo cohort), centring near 0.15-0.20 — roughly a quarter to a third of the ~0.60 SD claimed for Perry/Abecedarian, which is the intensity gap that blocks extrapolation from the boutique trials. The ~0.14 SD "average" formerly recorded here is an external synthesis figure, not a number this report states.
DOI / URL
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Language/literacy, 4-year-old cohort, END OF HEAD START YEAR (Exhibit 3a) | ITT SD | six significant direct-assessment impacts — PPVT (receptive vocabulary) +0.09, WJ III Letter-Word Identification +0.22, WJ III Spelling +0.15, WJ III Pre-Academic Skills +0.19, Color Identification +0.16, Letter Naming +0.25 — plus parent-reported Emergent Literacy Scale +0.31. LARGEST on the parent-reported measure, SMALLEST on the most g-loaded test. | mixed | spring 2003, end of the Head Start year | active-alternative | end-of-treatment | domain-skill |
| Math, pre-writing, grade promotion and teacher-rated school performance, 4-year-old cohort, ALL YEARS | ITT SD | zero — "No significant impacts were found for math skills, pre-writing, children's promotion, or teacher report of children's school accomplishments or abilities in any year." The 4-year-old cohort never showed a math impact at any wave. | mixed | all four waves | active-alternative | over-2yr | domain-skill |
| All cognitive outcomes, 4-year-old cohort, END OF KINDERGARTEN | ITT SD | zero — no significant cognitive impacts at all, one year after the programme ended | standardized | spring 2004, end of kindergarten | active-alternative | 1-2yr | domain-skill |
| Cognitive, 4-year-old cohort, END OF 1ST GRADE | ITT SD | PPVT +0.09, SUGGESTIVE ONLY (p<0.10) — the single cognitive impact at this wave, on the same measure that was smallest at the end of treatment. Everything else null. | standardized | spring 2005, end of 1st grade | active-alternative | over-2yr | domain-skill |
| Cognitive, 3-YEAR-OLD cohort, END OF HEAD START YEAR (Exhibit 3b) | ITT SD | broader than the 4-year-old cohort — PPVT +0.18, WJ III Letter-Word +0.26, CTOPPP Elision +0.10, Letter Naming +0.24, WJ III Pre-Academic Skills +0.22, pre-writing (McCarthy Draw-a-Design) +0.14, parent-reported Emergent Literacy +0.35, AND a math impact the older cohort never got — WJ III Applied Problems +0.15. | mixed | spring 2003, end of the age-3 Head Start year | active-alternative | end-of-treatment | domain-skill |
| Cognitive, 3-year-old cohort, END OF AGE-4 YEAR (second programme year) | ITT SD | only two of eight survive one more year — parent-reported Emergent Literacy +0.16 and CTOPPP Elision +0.15. Math, PPVT, Letter-Word, Letter Naming and Pre-Academic Skills all gone. Note the contrast is degraded here because controls gained Head Start access in year 2. | mixed | spring 2004, end of the age-4 year | active-alternative | end-of-treatment | domain-skill |
| Cognitive, 3-year-old cohort, END OF KINDERGARTEN — including the study's only unfavourable cognitive impact | ITT SD | Spanish Letter-Word Identification (Batería) +0.26 favourable; teacher-rated MATH ABILITY -0.19, UNFAVOURABLE — described by the report as "the only unfavourable cognitive impact found for either cohort as a whole in any year". It is not corroborated by the three direct math assessments, and the control group's kindergartens had significantly higher math proficiency. | mixed | spring 2005, end of kindergarten | active-alternative | 1-2yr | domain-skill |
| Cognitive, 3-year-old cohort, END OF 1ST GRADE | ITT SD | WJ III Oral Comprehension +0.08, SUGGESTIVE ONLY. Nothing else. | standardized | spring 2006, end of 1st grade | active-alternative | over-2yr | domain-skill |
| Social-emotional, 4-year-old cohort | ITT SD | zero on every measure in the Head Start year AND kindergarten. At the END OF 1ST GRADE the findings turn mixed and partly UNFAVOURABLE — teacher-reported socially reticent (shy/hesitant) behaviour higher in the Head Start group (moderate evidence) and more problematic student-teacher interactions (suggestive), against parent-reported less withdrawn behaviour (suggestive). | mixed | all four waves | active-alternative | over-2yr | non-cognitive |
| Social-emotional, 3-year-old cohort | ITT SD | the one domain where the younger cohort clearly moves, but ENTIRELY on parent report: strong evidence of less hyperactive behaviour and fewer total problem behaviours at the end of the Head Start year; suggestive better social skills/approaches to learning at age 4 and kindergarten plus moderate evidence of less hyperactivity at kindergarten; at 1st grade only parent-child closeness (moderate) and overall parent-child relationship (suggestive) remain. NO impacts on any teacher-reported social-emotional measure in kindergarten or 1st grade. | self-report | all four waves | active-alternative | over-2yr | non-cognitive |
| Received dental care (parent-reported) | ITT percentage points / SD | 4-year-old cohort +15pp (ES 0.31) at the end of the Head Start year; 3-year-old cohort +17pp (ES 0.33) at the end of the Head Start year and +10pp (ES 0.20) at the end of the age-4 year. Raw rates 73% vs 56% (4yo) and 68% vs 52% (3yo). Gone by 1st grade for both cohorts. Strong evidence (survives Benjamini-Hochberg) — the most robust short-run effect in the study, though the parent-reported Emergent Literacy effect sizes (0.31/0.35) are equally large. | self-report | end of the Head Start year (and age-4 year for the 3yo cohort) | active-alternative | end-of-treatment | health |
| Health insurance coverage and general health status (parent-reported) | ITT percentage points / SD | 4-year-old cohort — health insurance +4pp (ES 0.11) at kindergarten (suggestive) and again at 1ST GRADE (ES 0.11, moderate evidence); overall health excellent/good +5pp (ES 0.13) at kindergarten (suggestive). 3-year-old cohort — health status ES 0.11 at the Head Start year, insurance ES 0.14 at kindergarten, care for injury ES 0.10 at age 4, nothing at 1st grade. Health insurance for the 4yo cohort is the only favourable impact of ANY kind that both cohorts' 1st-grade waves retained. | self-report | kindergarten and 1st grade | active-alternative | over-2yr | health |
| Parenting practices | ITT SD | 4-year-old cohort — essentially nothing, one isolated impact (less use of time-out, ES -0.17, Head Start year). 3-year-old cohort — spanking -0.14 (Head Start year) and -0.09 (kindergarten), read to child +0.15, family cultural enrichment +0.18, authoritarian parenting -0.14 (age 4) and -0.11 (1st grade), time-out -0.13 (kindergarten) and -0.11 (1st grade). Parenting is the domain with the most durable 3-year-old-cohort impacts, and it is entirely parent self-report. | self-report | all four waves | active-alternative | over-2yr | non-cognitive |
| Absolute standing against national norms | percentile / percent | the gap is not closed. 4-year-old cohort PPVT in 2003: control group 27th percentile, Head Start group 31st. 3-year-old cohort: 29th vs 32nd. By the end of kindergarten 55% of the 4-year-old Head Start group and 58% of its controls could recognise all their letters, against 95% nationally (ECLS-K). | standardized | Head Start year and end of kindergarten | none | 1-2yr | domain-skill |
| Subgroup variation (exploratory, secondary — not confirmatory) | ITT, differential impacts | some subgroups retain cognitive impacts to 1st grade — 3-year-olds from high-risk households (five academic assessments at 1st grade), 3-year-olds in NON-URBAN settings (six language/literacy plus one math measure at 1st grade), 3-year-olds of parents with no depressive symptoms, and 4-year-olds of parents with mild depressive symptoms. Others are unfavourable — White 4-year-olds show unfavourable teacher-reported social-emotional impacts at 1st grade, and 3-year-olds of parents with MODERATE depressive symptoms show sustained negative impacts across cognitive, social-emotional and health domains including lower parent-reported grade promotion. The report labels all of this secondary and exploratory against ~10,000 comparisons. | mixed | all four waves | active-alternative | over-2yr | domain-skill |
Cited by
- Preschool at scale — Head Start, state pre-K, and what universal provision deliversmixedconf: highgc: low