Coaching for the Scholastic Aptitude Test: Further Synthesis and Appraisal

48 studies documented in 23 reports

US high-school students preparing for the SAT, 1950s to late 1980s

Meta-analysis of a literature the author describes as "rather poorly reported and designed without much attention to the issues discussed in reviews of the coaching literature" - the pooled estimate therefore inherits weak primary designs, which caps it at C. The important methodological choice is that published and unpublished studies were analysed SEPARATELY rather than pooled, which is how the publication-bias problem in this literature becomes visible instead of being averaged away.

Coaching raises SAT scores on average, but the average conceals large and systematic variability. In PUBLISHED COMPARISON studies - the subset with an actual control group - coached groups exceeded controls by 0.09 SD on SAT-V and 0.16 SD on SAT-M. Effects were consistently stronger for maths than verbal, which is the pattern every subsequent study reproduces and which fits the content-review mechanism rather than an ability mechanism. What moderated the effect is as informative as the effect: date of publication, whether ETS sponsored the study and used a comparison group, whether instruction included TEST PRACTICE AND ATTENTION TO TEST-TAKING SKILLS, and whether homework was assigned. The programme features that predict gains are procedural, not intellectual.

Medium. Most primary studies are observational with self-selected coached groups; the published-comparison subset is the least contaminated cell and is also the one with the smallest effects, which is the direction selection bias predicts.

Consistent with DerSimonian & Laird (1983), Messick & Jungeblut (1981), Powers & Rock (1999) and Briggs; the 0.09/0.16 SD figures are the field's most durable numbers.