Improving Reading Comprehension and Social Studies Knowledge in Middle School

27 eighth-grade US history classes, 5 teachers, 2 schools; 419 students assigned, 322-339 analysed

Eighth-graders in two near-urban Texas middle schools (53% white, 30% Hispanic, 23% FRL)

The first PACT trial and the origin of the whole programme. Two-stage randomisation: students randomly assigned to 27 social studies classes, then each teacher's classes randomly assigned to PACT or business-as-usual, so every teacher taught both conditions (within-teacher design, which controls teacher quality but risks contamination - teachers were explicitly trained on avoiding cross-contamination). Three 10-day text-based units on Colonial America, Road to Revolution and the Revolutionary War, delivered over 6-8 weeks. BOTH arms covered the same US history content; only the PACT arm used the five components. WWC reviewed it and rated it "meets evidence standards without reservations". Measure asymmetry is the thing to hold on to: the Gates-MacGinitie comprehension subtest is an independent standardized test, while both ASK subtests (knowledge acquisition, content reading comprehension) are researcher-developed by the intervention team, with the knowledge items lifted from released TAKS/MCAS/AP questions and aligned to the taught units. Only 2 schools and 5 teachers, so school-level generalisation is nil. Effect sizes below are the WWC's own recomputation; the authors report slightly different latent-variable estimates (g = 0.17 content acquisition, 0.29 content reading comprehension, 0.20 broad reading comprehension).

The only PACT trial in which standardized reading comprehension moved. WWC-computed effects: Gates-MacGinitie reading comprehension (independent standardized) ES = 0.21, p = .01; ASK reading comprehension in social studies (researcher-developed) ES = 0.35, p < .01; ASK knowledge acquisition (researcher-developed) ES = 0.31, p = .02. All three favour PACT and all three are statistically significant, and the WWC characterised both domains as showing statistically significant positive effects. The subsequent five trials by the same team reproduced the knowledge effect every time and reproduced the standardized-comprehension effect NEVER, which is the standard signature of a first-trial result that regressed. Note also the researcher-developed measures ran ~1.5x the standardized one within this single study, the same direction as the archive's 2x discount.

Low. Students were randomly assigned to classes and classes randomly assigned to condition within teacher, so genes and family background are balanced in expectation.

Directly replicated by the same team with 1,487 students in 85 classes (Vaughn et al. 2015), which reproduced the content-knowledge effect (g = 0.32) but found NO effect on either content-area or broad reading comprehension. Replicated again at 11th grade (Swanson et al. 2015, content g = 0.36, comprehension null) and at scale in a 48-school effectiveness trial (Roberts et al. 2023, knowledge 0.46, standardized comprehension 0.14 ns). All replications are internal to the developer team; PACT has never been independently evaluated.