Let's Think Secondary Science: Evaluation report and executive summary

53 secondary schools randomised (cost GBP 4,490 per school over two years, ~GBP 4 per pupil per year); 8,000+ Year 7 pupils; 5,882 in the primary analysis (3,020 intervention; 2,862 control)

England, three regions (north-east, midlands/north-west, south-west); whole Year 7 cohorts followed to end of Year 8 (ages 11-13).

Cluster RCT, school-level randomisation within region using matched pairs stratified on GCSE A*-C rate, FSM%, EAL% and roll. Intent-to-treat, multilevel model controlling for KS2 and gender. LTSS is Let's Think Forum's scaled adaptation of CASE: 19 one-hour lessons over two years (CASE had 30), delivered by science teachers IN PLACE OF normal science lessons, one training day per year plus three school support sessions per year. This is thinking-skills instruction, NOT inquiry or practical work — lessons deliberately do not address curriculum content. PRIMARY outcome: age-appropriate science test built from a Key Stage 3 SATs paper, administered by the school but scored centrally by trained markers to the national mark scheme. SECONDARY outcomes were the far-transfer test: GL Assessment Progress Test in English 13 and Progress in Maths 13 — commercially published, nationally standardised, wholly independent of the treatment. Because of logistics half the schools took English and half took maths, so each secondary outcome rests on only 17-20 schools and is underpowered; the transfer test was also pre-specified one-sided (only positive transfer could be detected). Attrition: 11% of schools dropped out, 26% of pupils missing outcome data — two padlocks removed, moderate security. Fidelity was poor: a quarter of teachers had taught 6 or fewer of the 19 lessons.

The independent effectiveness RCT of the CASE approach found nothing. Science attainment (the programme's home turf) g = -0.01 (95% CI -0.09 to 0.04), weighted -0.02; per-protocol +0.01. The two far-transfer outcomes that made CASE famous moved the WRONG WAY on independent standardised tests: English g = -0.15 (CI -0.24 to -0.06), weighted -0.13; maths g = -0.11 (CI -0.23 to 0.01), weighted -0.13 (CI -0.25 to -0.02). The evaluators refuse to call this negative transfer because the test was one-sided and underpowered, but the point estimates are the exact opposite sign to Adey & Shayer's claimed +0.67 to +0.72. A single subgroup — girls, science, g = 0.10 — reached significance out of many tests and the evaluators explicitly discount it. Overall estimate: -1 month's progress (0 months under EEF's revised conversion). Cost £4,490 per school over two years, ~£3.99 per pupil per year.

None material: school-level randomisation with matched-pair stratification, so genes cannot differ systematically between arms. Differential pupil attrition (26% missing) is the only selection channel, and weighted sensitivity analyses moved nothing.

This IS the replication attempt — an independent, EEF-funded, well-powered effectiveness test of the developer-led CASE findings of Adey & Shayer (1990, 1993) and Shayer (1999). It fails to reproduce either the science effect or the far transfer. Developers can point to two defences — LTSS cut CASE from 30 lessons to 19, and fidelity was poor — and to the fact that the original far-transfer effect only appeared at GCSE, three years later, which this trial did not measure. The report itself flags the GCSE follow-up as future work; as of this writing no such follow-up has been published.