The Evidence on Teaching

Promoting Acceleration of Comprehension and Content Through Text in High School Social Studies Classes

Swanson E, Wanzek J, Vaughn S, Roberts G, Fall A-M · 2015

grade Brctdeveloper-ledreplicated
Sample
41 eleventh-grade US history classes (23 treatment), 14 teachers
Population
Eleventh-graders enrolled in US History in US high schools
Design
Within-teacher randomised efficacy trial extending PACT from grade 8 to grade 11, so it tests whether the middle-school pattern holds with older students studying the same US history content. Both arms got the same content from the same teachers; only the treatment arm used the PACT components. Same outcome battery as the eighth-grade trials: a researcher-developed content-acquisition test built from released state/AP items, a researcher-developed social studies reading comprehension measure, and an independent standardized general reading comprehension test. Delayed testing at 4 and 12 weeks. Read at ABSTRACT depth (paywalled), cross-checked against the developers' own narrative of the trial in Roberts et al. (2023), which reports the same figures; attrition, clustering adjustments and moderator detail have not been verified.
Key findings
The grade-11 result is the grade-8 result again, which is what makes the pattern credible. Content acquisition: ES = 0.36 at posttest, 0.22 at 4 weeks, 0.24 at 12 weeks - a durable gain in US history knowledge that was still ~two-thirds of its original size three months later. And on BOTH comprehension measures - the content-area one and the general standardized one - there were no statistically significant effects at all. Five of the six PACT trials now say the same thing: content-rich history instruction teaches history, and does not teach reading.
Genetic confound
Low. Classes randomised within teacher; genes balanced in expectation across arms.
Replication notes
A conceptual replication at a new grade level of Vaughn et al. (2013) and Vaughn et al. (2015), and it replicates the knowledge effect (0.36 vs 0.32) and the comprehension null. Internal to the developer team; PACT has never been independently evaluated.
DOI / URL
10.1080/19345747.2014.906011

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
US history content acquisition, posttesteffect size0.36 (statistically significant)researcher-designedimmediately after the unitsbusiness-as-usualend-of-treatmentdomain-skill
US history content acquisition, delayedeffect size0.22 at 4 weeks; 0.24 at 12 weeksresearcher-designed4 and 12 weeks after the unitsbusiness-as-usualunder-1yrdomain-skill
Social studies reading comprehension and general reading comprehensionsignificance testno statistically significant effects on either measuremixedimmediately after the unitsbusiness-as-usualend-of-treatmentnear-transfer

Cited by