The Evidence on Teaching

Cognitive acceleration through science education II: its effects and scope

Shayer, M. · 1999

grade Dquasi-experimentdeveloper-ledfailednumbers spot-checked
Sample
8 schools, 1,452 pupils, 63 classes for the Piagetian analysis; ~4,500 pupils for the KS3/GCSE survey; 17 control schools for the regression benchmark
Population
England 1991-1997; whole-year cohorts in secondary schools that adopted CASE after King's College professional development, plus five Sunderland LEA schools trained by a King's-trained trainer.
Design
Written as a reply to Jones & Gott (1998), who had judged CASE "not proven". There is no control group in the usual sense. Piagetian gains are computed against the CSMS national survey norms (Shayer et al. 1976), i.e. against what the same pupils' pre-test percentile predicts. The KS3 and GCSE analyses regress attainment on mean intake percentile across 17 non-CASE schools and score CASE schools as positive residuals from that line. Every school in the analysis chose to adopt CASE and to pay for two years of King's professional development; the residual therefore contains the whole effect of being the kind of school that does that. Shayer concedes the school selection rationale for the later Shayer (2000) sample is unclear. The Piagetian outcome measure is the intervention's own theory made into a test, which Shayer defends explicitly ("No apology is needed for using Piagetian tests"). No randomisation, no attrition accounting, no adjustment beyond intake percentile.
Key findings
Large effects everywhere, from a design that cannot support them. Piagetian pre-post gains of 0.29σ to 1.26σ (school means ~0.9σ), ~30 percentile points against national norms. KS3 1995: CASE schools' percentage at National Curriculum level 6+ far exceeded regression predictions in science (e.g. 47.9% obtained vs 8.4% predicted), mathematics (all 12 schools significant) and English (9 of 11 schools significant). GCSE 1997 %C+ likewise exceeded prediction in science, maths and English for the five 11+ schools. Shayer's own argument for far transfer is the English result: "it just isn't plausible to believe that science process skills could transfer to English", therefore CASE must have raised general learning ability. He also retracts an earlier claim — the bimodality reported in Shayer & Adey (1993) "has to be attributed to small-sample variation".
Genetic confound
High. Self-selected adopter schools benchmarked against a regression line from 17 other schools, adjusted only for mean intake Piagetian percentile. School intake, parental selection, and the correlated decision to invest in two years of staff development all load onto the residual.
Replication notes
Failed under randomisation. The EEF effectiveness trial of the scaled programme (Hanley et al. 2016) randomised 53 schools and found g = -0.01 on science and negative point estimates on independent standardised English and maths tests. This paper is the strongest version of the pro-transfer case in terms of sample size, and it is a value-added residual analysis of volunteer schools.
DOI / URL
10.1080/095006999290345

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Piagetian Reasoning Tasks, CASE schools vs CSMS national norm-predicted gaind0.29 to 1.26 (school means ~0.9)researcher-designedend of two-year intervention (Year 8 or Year 9)noneend-of-treatmentdomain-skill
KS3 science, % at NC level 6+, obtained vs regression-predicted from intakepercentage points above predictione.g. 47.9% vs 8.4% predicted (school 1); all 7 CASE schools significantstandardizedage 14, one year after the interventionnone1-2yrdomain-skill
KS3 mathematics, % at NC level 6+ vs prediction — transfer claimpercentage points above predictione.g. 39.7% vs 13.0% predicted; all 12 schools significantstandardizedage 14none1-2yrfar-transfer
KS3 English, % at NC level 6+ vs prediction — the load-bearing far-transfer claimpercentage points above prediction9 of 11 schools significant (2 non-significant); e.g. 50% vs 11.9% predictedstandardizedage 14none1-2yrfar-transfer
GCSE English %C+ vs prediction, 11+ CASE schoolspercentage points above prediction4 of 5 schools significant (school 1 n.s.); e.g. 74.2% vs 38.9% predictedstandardizedage 16, three years after the interventionnoneover-2yrfar-transfer

Cited by