Efficacy of Schoolwide Programs to Promote Social and Character Development and Reduce Problem Behavior in Elementary School Children. Report From the Social and Character Development Research Program

84 schools across seven sites (10-14 schools each), about 6,600 students followed from grade 3 in autumn 2004 to grade 5 in spring 2007

United States; diverse locales; elementary school children.

The strongest null in the social-and-character-development literature and the most important single study in this topic. School-level random assignment via stratified pairwise matching, with control schools continuing their own standard activities. Critically, the impact analyses were conducted by Mathematica Policy Research under a separate competitive contract - the report states that the developers and implementers of these interventions did not conduct the impact analyses. The seven programmes include four flagships that appear in every SEL meta-analysis - Positive Action, PATHS, the 4Rs Program and Second Step. Minimum detectable effects for the combined analysis were 0.03 to 0.23 SD, more than three quarters of them below 0.10. One caveat must travel with this study - the Academics domain was PLANNED to include grades and standardized test scores but could not, because research teams could not obtain usable data, so the two academic outcomes are teacher-reported.

Across the combined-programme year-by-year analysis, 2 of 60 estimated impacts were statistically significant against 3 expected by chance, and both were on a teacher-reported measure of student support for teachers (0.12 and 0.16). None of the other 58 reached even substantive importance. At domain level there were 2 NEGATIVE significant impacts on social and emotional competence in years 2 and 3. For the individual programmes, 16 of 420 tests were significant against 21 expected by chance - 9 beneficial and 7 detrimental. In growth-curve analysis, 0 of 18 combined-programme impacts were significant with all effect sizes below 0.07, and 6 of 126 individual-programme impacts reached significance, exactly the chance rate, of which 2 were beneficial and 4 detrimental. For Positive Action's Chicago site specifically, none of the 18 estimated impacts on child-outcome trajectories was significant, with effect sizes of 0.00 to 0.09. Of five significant fidelity-outcome associations, FOUR were detrimental associations with low fidelity rather than beneficial associations with high fidelity. The report's conclusion is that on average the seven programmes did not improve students' social and emotional competence, behaviour, academic achievement, or student and teacher perceptions of school climate.

Low. School-level randomisation with stratified matching.

This IS the replication that failed, and it failed seven programmes simultaneously under one independent analyst. Several site teams published separate, far more favourable programme-specific papers on the same schools.