Thinking, Doing, Talking Science (second re-grant – a two-armed, cluster randomised trial): Evaluation report

180 schools randomised (168 contributing baseline data); 6,107 Year 5 pupils analysed (3,175 intervention, 2,932 control); FSM subgroup 1,852

Year 5 (age 9-10) pupils in English primary schools, Cohort 1 delivery 2022-23; areas deliberately non-overlapping with the earlier TDTS trials.

THIRD trial of the same programme, and the second independent effectiveness trial — run by York Trials Unit, co-funded by EEF and Wellcome. The developers' hypothesis after the 2018 null was that the train-the-trainer chain had been too thin, so Science Oxford built a more comprehensive accredited trainer programme, separately evaluated (Standley et al. 2023) before this trial began. Delivery was four and a half face-to-face days across five points in the year with gap tasks. The primary outcome was a Year 5 Science Assessment — a slightly modified version of the SAME instrument used in the 2015 efficacy and 2018 effectiveness trials — so measure type is held constant across all three trials in the chain. Implementation was strong: high compliance and fidelity, lesson observations confirmed practice change in line with the theory of change, pupils reported more discussion and practical work and less writing. About 16% of starting pupils were absent at testing. EEF security: HIGH. Observed ICC 0.10, MDES about 0.14. Cost GBP 9,210.90 per school over three years, about GBP 71.40 per child per year.

Hedges g = 0.02 (95% CI -0.07 to 0.12), p=0.66, n=6,107 — zero months' additional progress. FSM pupils g = 0.00 (-0.11 to 0.10), p=0.96, n=1,852 — also zero. Attitudes to science improved by a small amount, replicating the 2018 attitudes finding. The evaluators are explicit that teachers DID change practice as intended and pupils DID report higher engagement and self-efficacy, and that "these factors did not translate into improvements in pupils' science attainment at the end of Year 5". Reading the chain end to end: developer-delivered efficacy +0.22 (n=655) → independent effectiveness +0.01 (n=7,806) → independent effectiveness with a REPAIRED training model +0.02 (n=6,107). Fixing the implementation mechanism the developers blamed for the first null did not recover the effect. Because the outcome instrument is essentially constant across all three, this chain isolates efficacy-to-effectiveness decay from measure-type inflation.

Low. Large well-powered cluster RCT, independent evaluation, small baseline imbalance controlled in analysis.

Directly replicates Kitmitto et al. 2018 and directly fails Hanley, Slavin & Elliott 2015. An addendum reporting Cohort 2 (a "soak" year after teachers had embedded the approach) plus a longitudinal follow-up of Cohort 1 into Year 6 with a Year 6 science assessment and KS2 reading and maths was promised for 2025 and is not yet recorded here.