Nationally Normed Elementary Achievement Testing in America's Public Schools: How All 50 States Are Above the National Average
Cannell JJ · 1988
grade Dcritiqueindependentreplicated
Sample
reported results from all 50 US states plus large numbers of districts
Population
US public elementary schools using commercially published, nationally normed achievement batteries
Design
A survey-and-critique by a physician outside the measurement field, published in the measurement field's own journal with an editorial framing that treats it as a challenge rather than as a finished analysis. Methodologically it is a compilation of reported results, not a designed study - grade D. Its importance is entirely in the arithmetic contradiction it surfaced, which professionals then took up and quantified (see Linn, Graue & Sanders 1990 and Shepard 1990).
Key findings
All fifty states reported above-average performance on nationally normed achievement tests. This is arithmetically impossible against a live national norm, so something about the reported percentiles was wrong. The editorial note accompanying the article concedes the substance - "increased achievement is by no means the whole explanation for this finding" - and the subsequent literature attributed it to stale norms, teaching to the specific form, and non-representative local administration. The relevance to this archive is direct rather than historical: a homeschooling family who tests with a commercially normed battery is reading a percentile against exactly this kind of norm sample, and the "Lake Wobegon effect" is the reason a flattering percentile from a publisher-normed test is worth less than it appears.
Genetic confound
Not applicable - a critique of reported norm-referenced statistics.
Replication notes
Independently confirmed and quantified by Linn, Graue & Sanders (1990) and analysed by Shepard (1990); this is one of the few grade-D sources in the archive whose central claim was subsequently reproduced by better work.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| States reporting above-average performance on nationally normed tests | count | all 50 states | standardized | late 1980s reporting cycle | none | not-applicable | domain-skill |
| Whether rising achievement explains the pattern | editorial assessment | "increased achievement is by no means the whole explanation" | standardized | not applicable | none | not-applicable | domain-skill |
Cited by
- What a standardized achievement score does and does not licensemixedconf: mediumgc: low