The Evidence on Teaching

Evaluation of philosophy for children: an effectiveness trial

Lord, P., Dirie, A., Kettlewell, K., & Styles, B. · 2021

grade Arctindependentfailednumbers spot-checked
Sample
198 primary schools randomised (75 intervention, 123 control); 7,677 Year 6 pupils in the whole-cohort reading analysis, 3,358 in the FSM primary analysis
Population
England; whole primary schools, outcomes measured on Year 6 pupils (age 10-11), primary focus on pupils eligible for free school meals.
Design
Not a science programme — recorded because it is the cleanest test in existence of the mechanism CASE claims. A general thinking-skills programme (SAPERE's P4C: weekly one-hour enquiry sessions, whole-school training, Bronze/Silver/Gold accreditation) delivered for two years, with the far-transfer question asked directly: does teaching children to reason move their attainment in reading and maths? Two-arm cluster RCT with school-level randomisation; outcomes taken from the National Pupil Database (Key Stage 2 national tests) so the measure is fully independent of the deliverers and of the evaluators; very high (5-padlock) security; well powered for a small effect on FSM attainment. Fidelity was imperfect — 35 of 75 intervention schools were not implementing at the expected level after two years, and six not at all — the same defence available to CASE. The social-and-communication outcome used two single non-validated survey items carried over from the earlier trial, which the evaluators criticise.
Key findings
The 2015 EEF efficacy trial of P4C (Gorard, Siddiqui & See) reported about two additional months' progress in reading and maths — a far-transfer result of the same family as CASE's. The independent effectiveness re-trial found nothing: KS2 reading g = 0.02 (95% CI -0.09 to 0.13) for FSM pupils and 0.01 (-0.08 to 0.11) for the whole cohort; KS2 maths g = 0.05 (-0.06 to 0.16) FSM and 0.04 (-0.06 to 0.14) whole cohort; 0 months' progress on every measure. The two social-skills items gave -0.06 and +0.13, neither significant. Teacher perception was strongly positive (96% believed pupils' respect for others' opinions improved, 91% their ability to question and reason) while the attainment measures moved zero — a clean demonstration of why perceived reasoning gains are not evidence of transfer. Cost £13.50 per pupil per year.
Genetic confound
Low: school-level randomisation, national-test outcomes from administrative data, very high security rating.
Replication notes
This IS the failed replication. Two of the three flagship programmes in the Higgins et al. (2005) thinking-skills meta-analysis — P4C and CASE — have now been randomised at scale by the same What Works Centre, and both produced ~0 on independent standardised attainment. That is the pattern the archive should record about thinking-skills far transfer generally, not just about science.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
KS2 reading, FSM-eligible Year 6 pupils (primary outcome)Hedges g0.02standardizedend of Year 6, after two years of deliverybusiness-as-usualend-of-treatmentfar-transfer
KS2 reading, whole Year 6 cohortHedges g0.01standardizedend of Year 6business-as-usualend-of-treatmentfar-transfer
KS2 mathematics, whole Year 6 cohortHedges g0.04standardizedend of Year 6business-as-usualend-of-treatmentfar-transfer
Social and communication skills (two single non-validated survey items)Hedges g-0.06 and 0.13 (neither significant)researcher-designedend of Year 6business-as-usualend-of-treatmentnon-cognitive
Prior efficacy trial (Gorard, Siddiqui & See 2015) reading and mathsmonths' progress+2standardizedend of interventionbusiness-as-usualend-of-treatmentfar-transfer