Effectiveness of Cognitive Tutor Algebra I at Scale (RAND RCT)
Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. · 2014
grade Brctindependentmixed
Sample
73 high schools / 11,066 students (+ middle school arm)
Population
US Algebra I students, 7 states.
Design
Pair-matched school RCT of Carnegie Learning's adaptive mastery-gated blended curriculum. Implementation logs show teachers routinely OVERRODE the mastery pacing.
Key findings
The largest adaptive-mastery-platform RCT: nothing in year 1, a fragile +0.21 in year 2, and a significantly negative companion geometry trial. Anchors the honest range for mastery software at 0 to +0.2 on standardized tests — and mastery gating was widely bypassed by teachers in practice.
Genetic confound
Minimal (randomized).
Replication notes
Year-2 positive not replicated in year 1, the middle-school arm, or the geometry RCT; WWC overall rating 'mixed effects.'
DOI / URL
10.3102/0162373713507480
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Algebra proficiency, year 1 | ES | ~0 (ns) | standardized | end of course | business-as-usual | end-of-treatment | domain-skill |
| Algebra proficiency, year 2 | ES | +0.21 (p<.05; not robust to WWC multiplicity adjustment) | standardized | end of course | business-as-usual | end-of-treatment | domain-skill |
| Cognitive Tutor Geometry companion RCT | improvement index | significantly NEGATIVE (-8 percentile points) | standardized | end of course | business-as-usual | end-of-treatment | domain-skill |
Cited by
- Does educational technology raise learning — CAI, adaptive software, devices, screens, and AI tutors?mixedconf: mediumgc: low
- Mastery learning (teach → test → reteach to criterion → advance)mixedconf: highgc: low