The Evidence on Teaching

Comparing State and District Test Results to National Norms: The Validity of Claims That "Everyone Is Above Average"

Linn RL, Graue ME, Sanders NM · 1990

grade Cquasi-experimentindependentreplicated
Sample
reported achievement results from US states and a large sample of districts
Population
US public elementary and middle schools using nationally normed commercial achievement batteries
Design
The professional measurement community's audit of Cannell's claim - a compilation and analysis of reported state and district results against the publishers' norms, assessing which candidate explanations survive. Observational by construction, and it can attribute the inflation only by elimination rather than by manipulation, which is why it sits at C.
Key findings
The "everyone is above average" pattern is real, and it is not a data-entry artefact. The analysis takes seriously the question the archive needs answered - are these results "inflated and misleading?" - and identifies the factors that generate the abundance of above-average scores, principally the age of the publisher's norm sample and the local conditions of administration rather than genuine achievement growth. The operational lesson for reading a single child's percentile off a commercial battery: the norm you are being compared against may be old, may have been collected under different administration conditions, and may not be the population the score report implies.
Genetic confound
Not applicable - an audit of reported norm-referenced statistics.
Replication notes
Confirms Cannell (1988) independently and is itself consistent with Shepard's (1990) decomposition in the same issue; the phenomenon is settled, its exact decomposition less so.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Validity of above-national-average claims from commercially normed testsauditpattern confirmed and attributed substantially to norm age and administration conditions rather than achievementstandardizedlate 1980s reporting cyclenonenot-applicabledomain-skill

Cited by