Cognitive acceleration through science education II: its effects and scope
Shayer, M. · 1999
grade Dquasi-experimentdeveloper-ledfailednumbers spot-checked
Sample
8 schools, 1,452 pupils, 63 classes for the Piagetian analysis; ~4,500 pupils for the KS3/GCSE survey; 17 control schools for the regression benchmark
Population
England 1991-1997; whole-year cohorts in secondary schools that adopted CASE after King's College professional development, plus five Sunderland LEA schools trained by a King's-trained trainer.
Design
Written as a reply to Jones & Gott (1998), who had judged CASE "not proven". There is no control group in the usual sense. Piagetian gains are computed against the CSMS national survey norms (Shayer et al. 1976), i.e. against what the same pupils' pre-test percentile predicts. The KS3 and GCSE analyses regress attainment on mean intake percentile across 17 non-CASE schools and score CASE schools as positive residuals from that line. Every school in the analysis chose to adopt CASE and to pay for two years of King's professional development; the residual therefore contains the whole effect of being the kind of school that does that. Shayer concedes the school selection rationale for the later Shayer (2000) sample is unclear. The Piagetian outcome measure is the intervention's own theory made into a test, which Shayer defends explicitly ("No apology is needed for using Piagetian tests"). No randomisation, no attrition accounting, no adjustment beyond intake percentile.
Key findings
Large effects everywhere, from a design that cannot support them. Piagetian pre-post gains of 0.29σ to 1.26σ (school means ~0.9σ), ~30 percentile points against national norms. KS3 1995: CASE schools' percentage at National Curriculum level 6+ far exceeded regression predictions in science (e.g. 47.9% obtained vs 8.4% predicted), mathematics (all 12 schools significant) and English (9 of 11 schools significant). GCSE 1997 %C+ likewise exceeded prediction in science, maths and English for the five 11+ schools. Shayer's own argument for far transfer is the English result: "it just isn't plausible to believe that science process skills could transfer to English", therefore CASE must have raised general learning ability. He also retracts an earlier claim — the bimodality reported in Shayer & Adey (1993) "has to be attributed to small-sample variation".
Genetic confound
High. Self-selected adopter schools benchmarked against a regression line from 17 other schools, adjusted only for mean intake Piagetian percentile. School intake, parental selection, and the correlated decision to invest in two years of staff development all load onto the residual.
Replication notes
Failed under randomisation. The EEF effectiveness trial of the scaled programme (Hanley et al. 2016) randomised 53 schools and found g = -0.01 on science and negative point estimates on independent standardised English and maths tests. This paper is the strongest version of the pro-transfer case in terms of sample size, and it is a value-added residual analysis of volunteer schools.
DOI / URL
10.1080/095006999290345
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Piagetian Reasoning Tasks, CASE schools vs CSMS national norm-predicted gain | d | 0.29 to 1.26 (school means ~0.9) | researcher-designed | end of two-year intervention (Year 8 or Year 9) | none | end-of-treatment | domain-skill |
| KS3 science, % at NC level 6+, obtained vs regression-predicted from intake | percentage points above prediction | e.g. 47.9% vs 8.4% predicted (school 1); all 7 CASE schools significant | standardized | age 14, one year after the intervention | none | 1-2yr | domain-skill |
| KS3 mathematics, % at NC level 6+ vs prediction — transfer claim | percentage points above prediction | e.g. 39.7% vs 13.0% predicted; all 12 schools significant | standardized | age 14 | none | 1-2yr | far-transfer |
| KS3 English, % at NC level 6+ vs prediction — the load-bearing far-transfer claim | percentage points above prediction | 9 of 11 schools significant (2 non-significant); e.g. 50% vs 11.9% predicted | standardized | age 14 | none | 1-2yr | far-transfer |
| GCSE English %C+ vs prediction, 11+ CASE schools | percentage points above prediction | 4 of 5 schools significant (school 1 n.s.); e.g. 74.2% vs 38.9% predicted | standardized | age 16, three years after the intervention | none | over-2yr | far-transfer |
Cited by
- Teaching students to think scientifically — does it transfer?no effectconf: mediumgc: low