Online Mathematics Homework Increases Student Achievement

2,850 seventh-grade students in 43 schools; the analytic sample uses 22 randomly assigned school pairs

Seventh graders in Maine, USA

AERA Open 2(4), 1-12. A school-level cluster randomised trial - the strongest design applied to homework anywhere in this topic - with an INDEPENDENT standardized outcome (the TerraNova end-of-year mathematics assessment). THE CRITICAL SCOPING POINT: THIS IS NOT HOMEWORK VERSUS NO HOMEWORK. Treatment schools used the ASSISTments platform, which gives students immediate feedback and hints and gives teachers organised reports, plus teacher training; control schools 'continued with existing homework practices'. Homework quantity is held roughly constant and the FORMAT changes. Independence is recorded as developer-involved because the ASSISTments platform is an academic product and members of the evaluation ecosystem are close to it; the evaluation itself was conducted by SRI International.

Treatment raised end-of-year mathematics achievement by 8.835 scale points, t(20) = 2.992, p = .007, an effect size of 0.18 SD (Hedges's g), with the largest benefits for students with low prior mathematics achievement. Read against the rest of this topic, the number is instructive rather than vindicating: changing WHAT the homework is - adding immediate feedback, hints and teacher visibility - while holding quantity constant produces a well-identified 0.18 SD on an independent test, which is larger and far better identified than anything the quantity literature can offer at this age.

Low. Randomisation is at the school level within matched pairs, so child and family traits are balanced in expectation, and the outcome is an externally administered standardized test rather than a teacher grade.

Unreplicated at this scale. It sits alongside Rohrer and colleagues' preregistered cluster RCT of interleaved mathematics practice, which changed only the ORDERING of the same practice problems and produced d = 0.83 on a delayed unannounced test. Two independent randomised trials, both finding that the structure of the practice set moves achievement, against a quantity literature that cannot produce one clean student-randomised positive.