Improving the Targeting of Treatment

two systems: a large urban community-college system (~70,000 first-time degree-seekers, 2004-07 cohorts; analysis n = 37,813 maths and 34,697 English) and a statewide system of 50+ colleges (~49,000 students, 2 cohorts)

US community-college entrants placed by standardized placement examination (COMPASS and similar), late adolescence into adulthood

A predictive-validity simulation on very large administrative datasets, not an experiment in placement policy - it asks what would have happened under alternative screening rules given observed outcomes, and the "severe misplacement" definition depends on a predicted-grades model (predicted B-or-better versus predicted to fail). Graded B for the scale, the two independent systems, and the fact that the outcome is real subsequent course performance rather than another test score. The age range sits above this archive's K-12 core, so the numbers transfer as a mechanism rather than as a parameter.

A single cut-score-based placement test misplaces roughly a quarter to a third of the students it sorts. About one in four maths test-takers and one in three English test-takers are SEVERELY mis-assigned. The errors are strongly asymmetric: UNDER-placement (held back into remediation despite a predicted B or better) is two to six times more common than over-placement. In the urban system, 18.5% of maths test-takers and 28.9% of English test-takers were placed into remediation despite predicted B-or-better performance. Substituting high-school transcript information for the test, holding the remediation rate constant, removes four to eight severe misplacements per hundred students - up to a 30% reduction - and raises the success rate among those placed into college-level work by about 10 percentage points (76% to 89% C-or-better in one system). Adding the test on top of transcript data adds little. The general lesson for an intake tool: a single sitting is a weak placement instrument, an accumulated performance record beats it, and the errors it makes are systematically the errors of holding capable students back.

Low relevance for the misplacement estimate, which is about the instrument's accuracy against an observed criterion rather than about differences between students.

The two systems in the paper agree with each other, but no independent replication on a different national context was identified. The finding drove real policy change (the discontinuation of COMPASS), which is corroboration of a different kind.