Calderdale Excellence Partnership: IPEELL. Evaluation report and executive summary
Torgerson CJ, Ainsworth H, Bell K, Elliott L, Fountain I, Gascoine L, Hewitt CE, Kasim A, Kokotsaki D, Torgerson DJ · 2018
grade Brctindependentfailed
Sample
One-year trial 84 schools / 2,682 children (2,465 analysed); two-year trial 83 schools / 2,762 children (2,196 analysed)
Population
Year 6 pupils (one-year trial) and Year 5 into Year 6 pupils (two-year trial) in English primary schools; IPEELL was delivered to all children in those year groups
Design
Independently evaluated cluster RCT (Durham/York), commissioned by the EEF as the EFFECTIVENESS trial of the SRSD-derived IPEELL writing programme after the 2014 efficacy trial produced ~+0.74. Tested under "real world" conditions - trainers newly recruited rather than the programme developers, delivered across the full prior-attainment range rather than to low attainers only, and in Years 5-6 rather than across the Year 6-7 transition. Outcome for the one-year trial was the KS2 writing outcome (which had been recategorised into three bands during set-up, reducing sensitivity); the two-year trial used a bespoke writing test built from historical KS2 writing questions to recover a continuous outcome. EEF security: very high (one-year), high (two-year; reduced because >20% of starting pupils lacked post-test data). Cost ~GBP 36 per pupil per year over three years.
Key findings
Two years of IPEELL: ES = 0.11 (95% CI -0.13 to 0.34), 2 months' progress, n=2,196 - NOT statistically significant. FSM pupils: 0.14 (-0.14 to 0.41), n=811. One year of IPEELL: ES = -0.09 (-0.34 to 0.15), -1 month, n=2,465 - also not significant, and pointing the wrong way. FSM one-year: -0.07 (-0.38 to 0.23). Low prior attainers in the two-year trial did better (3 months' additional progress) on a smaller and less secure sample. The spillover finding is the one most worth carrying: "Pupils who used IPEELL for two years made SIGNIFICANTLY LESS PROGRESS IN READING, SPELLING AND MATHS compared to pupils who did not", with the same pattern (reading significant) in the one-year trial - the evaluators' hypothesis being that class time was directed away from those subjects and towards writing. The evaluators state plainly that "a previous EEF-funded trial of IPEELL in England showed positive impacts. In contrast, this project did not find evidence of consistent positive effects." Training quality was also flagged as weak, delivered by inexperienced trainers.
Genetic confound
Low. Large cluster-randomised design, independent evaluation, national outcome measures for the one-year trial. Residual exposure is the >20% missing post-test data in the two-year trial.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Writing, two years of IPEELL | effect size (95% CI) | 0.11 (-0.13 to 0.34); 2 months, not significant | researcher-designed | end of Year 6 after 2 years | business-as-usual | end-of-treatment | domain-skill |
| Writing, one year of IPEELL | effect size (95% CI) | -0.09 (-0.34 to 0.15); -1 month, not significant | standardized | end of Year 6 after 1 year | business-as-usual | end-of-treatment | domain-skill |
| Writing, FSM pupils (two-year / one-year) | effect size (95% CI) | 0.14 (-0.14 to 0.41) / -0.07 (-0.38 to 0.23) | researcher-designed | end of Year 6 | business-as-usual | end-of-treatment | domain-skill |
| Reading, spelling and maths spillover, two-year trial | group comparison | significantly LESS progress than comparison pupils in all three | standardized | end of Year 6 | business-as-usual | end-of-treatment | domain-skill |
| Writing, low prior attainers, two-year trial | months' progress | 3 months' additional progress (smaller sample, lower security) | researcher-designed | end of Year 6 | business-as-usual | end-of-treatment | domain-skill |
Cited by
- Can capitalisation, punctuation and usage be taught directly as their own subject?mixedconf: mediumgc: medium
- Does strategy instruction (SRSD, the writing process) improve writing?moderate supportconf: mediumgc: low
- Does teaching spelling work, and what does it transfer to?moderate supportconf: mediumgc: low