Inflated Test Score Gains: Is the Problem Old Norms or Teaching the Test?

none of its own - analysis of reported state and district results and of the candidate explanations

US public schools using nationally normed commercial achievement batteries

An analytic essay in the measurement field's practitioner journal, without original data collection, so grade D. It is recorded because it poses the decomposition question directly - is the Lake Wobegon effect stale norms, or is it teaching the test? - and because its answer is "both, and they interact", which is the answer the later score-inflation literature confirmed with better designs.

The two explanations for spurious score gains are not alternatives. Old norms inflate reported percentiles on their own, because the reference group is a cohort that no longer exists. Teaching to the specific test form inflates the raw scores. And crucially, teaching the test CONTAMINATES THE NORMS THEMSELVES, because the norming sample was drawn from schools that also practise on the form - so the reference point drifts in the same direction as the scores being compared to it. Shepard poses the practical question the archive inherits: what alternatives protect the integrity of instruction and the validity of scores at the same time.

Not applicable - a critique of score reporting.

The teaching-to-the-test half was subsequently established with audit-test designs in Kentucky, Texas and Chicago; the stale-norms half was quantified by Linn, Graue & Sanders in the same journal issue.