Tutoring — the honest effect, the Bloom 2-sigma myth, and what survives scale

Verdict

Tutoring is the best-replicated positive intervention in this database — and the gap between its folklore and its evidence is the most instructive in education. Bloom's 2-sigma is a myth: a compound artifact of researcher-built tests on 3-week novel units, tutoring fused with retest-to-criterion mastery (the mastery arm alone was ~1.1σ), specially trained tutors, extra time, and a higher criterion in the tutored arm — never replicated in 160+ controlled studies since, and Bloom's own cited source put tutoring at d=0.40. The real numbers form a clean gradient:

• Efficacy RCTs (small, well-run): pooled ≈0.29 (published Nickow meta; 0/96 studies reach 2.0)
• Well-run at scale (US, standardized tests, 400+ students): 0.16–0.21
• Best replicated program (Saga Chicago, daily in-school 2:1): 0.18–0.40, with rare measured persistence (+0.23 SD two years later)
• Typical district/national scale (Nashville, ESSER districts, UK's £1B programme): 0.00–0.09

Even the honest number is large by education standards — one year of Saga roughly doubles a year of normal high-school math learning — which is why the verdict is strong-support. But the at-scale gradient is the school builder's central lesson: the model is not what fails at scale; implementation is. Programs that kept the bundle intact lost only ~18% of their effect at scale; programs in general lost ~42%; programs that went opt-in/after-school/low-dosage lost everything.

What the evidence shows

Design facts that survive across all of it:

• 1:1 is unnecessary. No advantage over small groups anywhere (Elbaum; Nickow; Saga's 2:1; Bhatt's 4:1 tech-rotation kept 0.23 at 30% lower cost). The binding ingredient is daily scheduled in-school dosage with an accountable adult, not the ratio.
• Frequency beats total hours: ≥3 sessions/week; once-weekly programs repeatedly fail.
• Paid beats volunteer (0.30–0.39 vs 0.17), though structured supervised volunteers manage ~0.30 on drilled subskills (Ritter).
• In-school beats after-school (~2×); opt-in models fail on take-up — the students who need it don't show up.
• Virtual is ~1/4–1/2 as effective at ~1/3 the cost (OnYourMark: 0.05–0.12 at $1,400) — a legitimate option where staffing fails, not a substitute.
• Software parity: step-based tutoring systems match human tutors in efficacy settings — but collapse to ~0.13 on broad standardized measures, and adaptive systems can harm low achievers.

Durability is tutoring's soft spot: Saga's ~70% retention at 1–2 years is the exception; the UK national programme showed ~no persistence, and the flagship 1:1 program (Reading Recovery) shows contested evidence of long-term reversal. Assume decay and plan re-dosing.

Hereditarian-lens assessment

Risk: low (randomized designs throughout). Three lens findings worth a founder's attention:

1. Bloom's equalization claim is refuted, not just his effect size. The falling aptitude-achievement correlation he celebrated is a ceiling artifact; heritability of achievement rises with age, and time-to-mastery ratios grow. Tutoring raises the mean for those tutored; it does not abolish ability differences. "Move anyone to the top 2%" was never real.
2. Human-tutoring RCTs cannot tell you tutoring "closes gaps" — ~90% sample only low performers, so no high-ability arm exists. What's measurable: it reliably raises low performers' domain skills.
3. Software tutoring can widen gaps (meta evidence of negative effects for low achievers, g=−0.18, while helping the general population) — a caution against assuming tech variants inherit human-tutoring's equity profile.
4. Sample-SD units on low-performer samples overstate population-SD effects somewhat — one more reason the honest planning number is ~0.2, not 0.4.

What critics say / limits

• Fryer's well-powered NYC high-dosage reading RCT was null at the mean — tutoring is more reliable in math than reading, and reading effects concentrate in early grades.
• Peer-reviewed studies pool higher than grey literature (0.45 vs 0.22) despite clean funnel tests — selective reporting can't be ruled out.
• Only 9 RCTs exist of programs serving ≥1,000 students; the at-scale bin is imprecise.
• One district program produced significantly negative math effects (plausibly displacing core instruction) — tutoring time comes from somewhere.

Practical guidance

• The recipe (each element evidence-moderated): groups of 2–4 · paid, trained, consistent tutors · scheduled period during the school day · 3–5 sessions/week · ≥30 actually-delivered hours · curriculum-aligned, data-informed sessions.
• Budget $1,400–4,800/student/yr depending on labor model; tech-rotation 4:1 (~$2,500–3,000) is the best current cost-effect compromise; structured virtual ($1,400) where staffing fails.
• Target it: biggest, most reliable effects for students behind grade level, math, and (for reading) early grades. It is the best-evidenced remediation tool in existence — not a whole-population strategy at 20× the cost of curriculum changes.
• Measure delivered dosage, not enrolled students — implementation is where every failed scale-up died.
• Assume fadeout without re-dosing; buy persistence by keeping students in the system across years (the Saga model) rather than one-shot pull-outs.

Open questions

• Whether AI tutors change the cost curve: step-based systems already match human efficacy on aligned measures, but no published causal evidence yet shows chatbot tutoring moving standardized achievement (claims of "2-sigma AI tutoring" recapitulate Bloom's error).
• Whether Saga-style persistence survives independent (non-developer) replication at scale.