The Evidence on Teaching

Multi-Level Meta-Analysis of Physical Activity Interventions During Childhood: Effects of Physical Activity on Cognition and Academic Achievement

Vasilopoulos F, Jeffrey H, Wu Y, Dumontheil I · 2023

grade Cmeta-analysisindependentnot-applicablenumbers spot-checked
Sample
92 randomized controlled trials, N = 25,334 typically developing children aged 5-12
Population
Typically developing children aged 5-12
Design
Three-level meta-analysis handling nested effect sizes within studies (270 effect sizes from 92 RCTs), with GRADE-style certainty ratings attached to each pooled estimate and moderator analyses for control-group type, duration, frequency and instructor qualification. Restricted to RCTs in typically developing children, which is the right scope for schooling. Publication bias was ASSESSED (LFK index and Doi plots) but never ADJUSTED for - there is no PET-PEESE or selection-model corrected estimate to report, so the pooled figures below are all uncorrected. The nearest thing to a corrected estimate is the influential-case-removed sensitivity analysis, which is recorded alongside each headline number. Only 18 of the 92 included trials were preregistered (the paper calls this 24%; 18/92 is 20%). Risk of bias was rated with the EPHPP tool and called "generally low", but 92% of studies did not blind outcome assessment or delivery (assessors were blinded in six studies), 79% of samples were not representative of the target population (16 of 92 were), and about half the trials were feasibility-sized. Only objective cognitive/academic instruments were eligible, so no self-report outcomes are pooled. Graded C as a meta-analysis of mixed-quality RCTs without publication-bias adjustment.
Key findings
FOUR of the paper's pooled estimates are positive and significant, and the certainty ratings are what separate them. The two large ones are the two least trustworthy: on-task behaviour g=1.04 (very low certainty, CI 0.08 to 2.00, 5 studies / 6 effect sizes) and creativity g=0.70 (low certainty, CI 0.20 to 1.20, p<0.01, 5 studies / 14 effect sizes). The two small ones are fluid intelligence g=0.16 (0.02 to 0.30, p=0.03, moderate certainty, 8 studies) and working memory g=0.18 (0.07 to 0.29, p=0.01, VERY LOW certainty, 30 studies). Against them sits the paper's only other moderate-certainty estimate, and it is a null: academic achievement g=0.14 (-0.04 to 0.31, p=0.14, 47 studies / 114 effect sizes, N=14,386). So the best-evidenced academic estimate in the paper is a null, and the best-evidenced cognitive estimate is a small positive on fluid intelligence. Decisive for how much weight the positives can carry: the authors report evidence of publication bias for all three of the outcomes they count as having significant summary effects (fluid intelligence, on-task behaviour, working memory) and say the results "should therefore be interpreted with caution" — AND for three more (inhibitory control, creativity and planning), so six of the nine outcomes assessed show funnel asymmetry, including every significant positive. No overall benefit was found for academic achievement, for combined executive function (g=0.09, -0.06 to 0.25, p=0.24, very low certainty), nor for attention (0.08), cognitive flexibility (0.05), planning (0.09) or inhibitory control, whose point estimate is negative (-0.06, -0.29 to 0.16). Language (0.08) and maths (0.15) are both null on their own. The moderator results the authors highlight — 3x/week, 7-10-week, 20-44-min PA-with-academic-instruction programmes helping mathematics (g=0.28 to 0.36), enriched programmes helping language (g=0.10), and MVPA/aerobic PA helping executive function (g=0.21 / 0.16) — are exactly the short-duration, small-sample subgroup estimates that bias correction removes elsewhere in this literature, and they are moderators of main effects that are themselves null. Note the inversion of the usual "long dose works better" intuition: the effects appear at 7-10 weeks and vanish over years (see A+PAAC at 3 years).
Genetic confound
Randomized corpus, so no genetic confounding of the treatment contrast.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Academic achievement (overall)Hedges' g, three-level model0.14, p=0.14 (95% CI -0.04 to 0.31) — NOT significant, moderate certainty, k=47, 114 ES, N=14,386; influential cases removed 0.10 (-0.08 to 0.28), p=0.26mixedend of interventionbusiness-as-usualend-of-treatmentfar-transfer
Mathematics achievementHedges' g0.15, p=0.13 (95% CI -0.05 to 0.34) — null, k=42; influential cases removed 0.17 (-0.03 to 0.37), p=0.10mixedend of interventionbusiness-as-usualend-of-treatmentfar-transfer
Language achievementHedges' g0.08, p=0.48 (95% CI -0.14 to 0.31) — null, k=32mixedend of interventionbusiness-as-usualend-of-treatmentfar-transfer
On-task behaviourHedges' g1.04, p=0.03 (95% CI 0.08 to 2.00) — VERY LOW certainty, k=5, 6 ES, N=804; I2 95.9%; publication bias detectedresearcher-designedend of intervention; classroom observationbusiness-as-usualend-of-treatmentbehaviour
CreativityHedges' g0.70, p<0.01 (95% CI 0.20 to 1.20) — LOW certainty, k=5, 14 ES, N=345; drops to 0.50 (0.36 to 0.65) with influential cases removed; publication bias detectedmixedfar transfer; end of interventionbusiness-as-usualend-of-treatmentfar-transfer
Fluid intelligenceHedges' g0.16, p=0.03 (95% CI 0.02 to 0.30) — moderate certainty, k=8, 9 ES, N=1,351; publication bias detectedstandardizedfar transfer; end of interventionbusiness-as-usualend-of-treatmentg
Working memoryHedges' g0.18, p=0.01 (95% CI 0.07 to 0.29) — very low certainty, k=30, 46 ES; influential cases removed 0.14 (0.05 to 0.24), p<0.01; publication bias detectedstandardizedfar transfer; end of interventionbusiness-as-usualend-of-treatmentfar-transfer
Executive function (combined)Hedges' g0.09, p=0.24 (95% CI -0.06 to 0.25) — null, very low certainty, k=45, 127 ES, N=8,448standardizedfar transfer; end of interventionbusiness-as-usualend-of-treatmentfar-transfer
Attention, inhibitory control, planning, cognitive flexibilityHedges' g per sub-domainall null: attention 0.08 (95% CI -0.08 to 0.24, p=0.32); cognitive flexibility 0.05 (-0.17 to 0.26, p=0.63); planning 0.09 (-0.37 to 0.55, p=0.62); inhibitory control NEGATIVE at -0.06 (-0.29 to 0.16, p=0.57)standardizedfar transfer; end of interventionbusiness-as-usualend-of-treatmentfar-transfer
Academic achievement in favourable moderator subgroups (main effect is null)Hedges' g, meta-regression subgroupsPA with academic instruction 0.31 (95% CI 0.10 to 0.51, n=47 ES); 3x/week 0.28 (0.07 to 0.50); 7-10 weeks 0.36 (0.11 to 0.60); 20-44 min sessions 0.27 (0.12 to 0.42); driven by MATHS (3x/week 0.28, p=0.01; 7-10 weeks 0.29, p=0.04; PA-with-instruction 0.32, p=0.01), except enriched PA which acts on language (0.10, p=0.04)mixedend of intervention; subgroup/moderator, not ITTbusiness-as-usualend-of-treatmentfar-transfer
Executive function in favourable moderator subgroups (main effect is null)Hedges' g, meta-regression subgroupsmoderate-to-vigorous intensity 0.21 (95% CI 0.07 to 0.34, n=63 ES); aerobic interventions 0.16 (0.03 to 0.28, n=33 ES); no EF sub-domain moderator effects survivedstandardizedend of intervention; subgroup/moderator, not ITTbusiness-as-usualend-of-treatmentfar-transfer
Publication-bias correctionLFK index / Doi plotsnone applied — bias was ASSESSED and found for 6 of 9 outcomes (fluid intelligence, on-task behaviour, working memory, inhibitory control, creativity, planning) but NO corrected pooled estimate (no trim-and-fill, PET-PEESE or selection model) is reported anywhere in the papermixednot applicablenonenot-applicableunclear

Cited by