The impact of schooling on academic achievement: Evidence from homeschooled and traditionally schooled students

74 children aged 5-10 — 37 home-schooled (25 structured, 12 unstructured) and 37 public school

Families in Nova Scotia and New Brunswick, Canada, recruited by community announcements, e-mail and radio. Home-schooled children had never attended public school from Grade 1; comparison children had never been home-schooled.

The methodologically best-designed comparison in the literature and still only n=74. Each home-schooled child was paired with a similar-age public-school child in the same geographical area (age match: mean difference 2 months, t(72)=0.012, p=.99), and every child was tested by the same trained experimenter with the same battery, mostly in the home. Maternal education and family income were entered as MANOVA covariates and were not significant; effect sizes shifted by <=.02. The structured/unstructured split was made post hoc, and the unstructured cell is 12 children. Outcome: Woodcock-Johnson Tests of Achievement (Revised, Form A), seven subtests, analysed as grade-level difference scores (actual minus predicted grade level).

Structured home-schoolers beat public-school children on all seven subtests (Hotelling's Trace .362, F(7,54)=2.79, p=.015, partial eta-squared .27). Subtest gaps in grade levels: Letter-Word 3.11 vs 1.38 (p=.002), Word Attack 4.89 vs 1.61 (p=.006), Social Science 1.59 vs 0.59 (p=.004), Science 2.61 vs 1.37 (p=.019), Humanities 0.99 vs -0.005 (p=.039), Calculation 0.94 vs 0.27 (p=.026), Passage Comprehension 2.56 vs 1.58 (p=.068, n.s.). Under Bonferroni (p<.007) only Letter-Word, Word Attack and Social Science survived. UNSTRUCTURED home-schoolers (unschoolers, n=12) scored BELOW public school on all seven measures, by 0.64 (Calculation) to 1.67 (Reading Comprehension) grade levels, none surviving Bonferroni; and below structured home-schoolers on all seven by 1.32 to 4.20 grade levels, five of seven surviving Bonferroni with eta-squared .15-.34. The public-school comparison children were themselves scoring above grade level, so this is not a weak counterfactual.

High. All three groups self-selected into a study about education; the authors concede structured-group scores "may be somewhat inflated" by volunteer bias. Covarying maternal education and income moves nothing, but those are weak proxies for the heritable traits that select a family into structured home education in the first place — and the same objection cuts against reading the unschooling result as causal harm.

Never replicated. Gaither 2017 calls its method the likely future direction of the field while flagging the small sample, the post-hoc structured/unstructured split, and the absence of any measure of how long children had been home-schooled. Neuman & Guterman 2016 show the structured/unstructured binary itself conflates two independent dimensions.