The Evidence on Teaching

Nationally Normed Elementary Achievement Testing in America's Public Schools: How All 50 States Are Above the National Average

Cannell JJ · 1988

grade Dcritiqueindependentreplicated
Sample
reported results from all 50 US states plus large numbers of districts
Population
US public elementary schools using commercially published, nationally normed achievement batteries
Design
A survey-and-critique by a physician outside the measurement field, published in the measurement field's own journal with an editorial framing that treats it as a challenge rather than as a finished analysis. Methodologically it is a compilation of reported results, not a designed study - grade D. Its importance is entirely in the arithmetic contradiction it surfaced, which professionals then took up and quantified (see Linn, Graue & Sanders 1990 and Shepard 1990).
Key findings
All fifty states reported above-average performance on nationally normed achievement tests. This is arithmetically impossible against a live national norm, so something about the reported percentiles was wrong. The editorial note accompanying the article concedes the substance - "increased achievement is by no means the whole explanation for this finding" - and the subsequent literature attributed it to stale norms, teaching to the specific form, and non-representative local administration. The relevance to this archive is direct rather than historical: a homeschooling family who tests with a commercially normed battery is reading a percentile against exactly this kind of norm sample, and the "Lake Wobegon effect" is the reason a flattering percentile from a publisher-normed test is worth less than it appears.
Genetic confound
Not applicable - a critique of reported norm-referenced statistics.
Replication notes
Independently confirmed and quantified by Linn, Graue & Sanders (1990) and analysed by Shepard (1990); this is one of the few grade-D sources in the archive whose central claim was subsequently reproduced by better work.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
States reporting above-average performance on nationally normed testscountall 50 statesstandardizedlate 1980s reporting cyclenonenot-applicabledomain-skill
Whether rising achievement explains the patterneditorial assessment"increased achievement is by no means the whole explanation"standardizednot applicablenonenot-applicabledomain-skill

Cited by