The Evidence on Teaching

THE EFFECTS OF INSTRUCTION AND PRACTICE IN CONTENT-INDEPENDENT TEST-TAKING TECHNIQUES UPON THE STANDARDIZED READING TEST SCORES OF SELECTED SECOND-GRADE STUDENTS

Callenbach C · 1973

grade Crctindependentunreplicated
Sample
24 relatively test-naive second-grade students, experimental vs control
Population
US second-grade students, described as relatively test-naive
Design
Small controlled trial (n=24) with an immediate and a four-month delayed posttest on an independent published measure (Stanford Reading Test). Tiny, unregistered, single-site, and one classroom's worth of children, so the point estimate is soft. What makes it worth its grade is the design logic - the treatment contained NO reading instruction whatsoever, so any movement on a reading test is by construction a change in test performance rather than in reading.
Key findings
Eight 30-minute sessions of purely content-independent test-taking technique - how to follow a separate answer sheet, how to pace, how to handle item formats - raised STANDARDIZED READING TEST scores by about a quarter of a standard deviation, significant immediately and still significant four months later. No reading was taught. This is the cleanest demonstration in the archive that a standardized score is a performance and not a capacity: roughly 0.25 SD of a young child's reading score can be moved without touching their reading. It cuts both ways for an intake tool - it is a reason to discount score gains after any test-familiarisation, and equally a reason to suspect that a test-naive child's FIRST score understates them.
Genetic confound
Low. Randomised-to-condition comparison of an instructional manipulation; genes cannot differ systematically between arms, and the outcome is an independent published test.
Replication notes
Not directly replicated, but consistent in direction and magnitude with the meta-analytic literature on coaching and test-taking-skills training for achievement tests.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Standardized reading test score, immediate posttestmean difference, SD units~0.25 SD, significantstandardizedone week after 4-week treatmentbusiness-as-usualend-of-treatmentdomain-skill
Standardized reading test score, delayed posttestmean difference, SD units~0.25 SD, still significantstandardizedfour months post-treatmentbusiness-as-usualunder-1yrdomain-skill
Reading skill actually taughtcontent of treatmentnone - treatment was content-independent test-taking technique onlystandardized8 x 30 min over 4 weeksbusiness-as-usualend-of-treatmentdomain-skill

Cited by