Evaluation of philosophy for children: an effectiveness trial
Lord, P., Dirie, A., Kettlewell, K., & Styles, B. · 2021
grade Arctindependentfailednumbers spot-checked
Sample
198 primary schools randomised (75 intervention, 123 control); 7,677 Year 6 pupils in the whole-cohort reading analysis, 3,358 in the FSM primary analysis
Population
England; whole primary schools, outcomes measured on Year 6 pupils (age 10-11), primary focus on pupils eligible for free school meals.
Design
Not a science programme — recorded because it is the cleanest test in existence of the mechanism CASE claims. A general thinking-skills programme (SAPERE's P4C: weekly one-hour enquiry sessions, whole-school training, Bronze/Silver/Gold accreditation) delivered for two years, with the far-transfer question asked directly: does teaching children to reason move their attainment in reading and maths? Two-arm cluster RCT with school-level randomisation; outcomes taken from the National Pupil Database (Key Stage 2 national tests) so the measure is fully independent of the deliverers and of the evaluators; very high (5-padlock) security; well powered for a small effect on FSM attainment. Fidelity was imperfect — 35 of 75 intervention schools were not implementing at the expected level after two years, and six not at all — the same defence available to CASE. The social-and-communication outcome used two single non-validated survey items carried over from the earlier trial, which the evaluators criticise.
Key findings
The 2015 EEF efficacy trial of P4C (Gorard, Siddiqui & See) reported about two additional months' progress in reading and maths — a far-transfer result of the same family as CASE's. The independent effectiveness re-trial found nothing: KS2 reading g = 0.02 (95% CI -0.09 to 0.13) for FSM pupils and 0.01 (-0.08 to 0.11) for the whole cohort; KS2 maths g = 0.05 (-0.06 to 0.16) FSM and 0.04 (-0.06 to 0.14) whole cohort; 0 months' progress on every measure. The two social-skills items gave -0.06 and +0.13, neither significant. Teacher perception was strongly positive (96% believed pupils' respect for others' opinions improved, 91% their ability to question and reason) while the attainment measures moved zero — a clean demonstration of why perceived reasoning gains are not evidence of transfer. Cost £13.50 per pupil per year.
Genetic confound
Low: school-level randomisation, national-test outcomes from administrative data, very high security rating.
Replication notes
This IS the failed replication. Two of the three flagship programmes in the Higgins et al. (2005) thinking-skills meta-analysis — P4C and CASE — have now been randomised at scale by the same What Works Centre, and both produced ~0 on independent standardised attainment. That is the pattern the archive should record about thinking-skills far transfer generally, not just about science.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| KS2 reading, FSM-eligible Year 6 pupils (primary outcome) | Hedges g | 0.02 | standardized | end of Year 6, after two years of delivery | business-as-usual | end-of-treatment | far-transfer |
| KS2 reading, whole Year 6 cohort | Hedges g | 0.01 | standardized | end of Year 6 | business-as-usual | end-of-treatment | far-transfer |
| KS2 mathematics, whole Year 6 cohort | Hedges g | 0.04 | standardized | end of Year 6 | business-as-usual | end-of-treatment | far-transfer |
| Social and communication skills (two single non-validated survey items) | Hedges g | -0.06 and 0.13 (neither significant) | researcher-designed | end of Year 6 | business-as-usual | end-of-treatment | non-cognitive |
| Prior efficacy trial (Gorard, Siddiqui & See 2015) reading and maths | months' progress | +2 | standardized | end of intervention | business-as-usual | end-of-treatment | far-transfer |