The Evidence on Teaching

A Splintered Vision

Schmidt, W. H., McKnight, C. C., Raizen, S. A., Jakwerth, P. M., Valverde, G. A., Wolfe, R. G., Britton, E. D., Bianchi, L. J., & Houang, R. T. · 1997

grade Dreviewindependentnot-applicable
Sample
628 textbooks and 491 curriculum guides from ~45-50 TIMSS countries, plus TIMSS 1995 teacher questionnaire data (US, Japan, Germany).
Population
US mathematics and science curriculum documents, textbooks and teacher self-reports, grades 4 and 8, benchmarked against other TIMSS countries.
Design
Full subtitle: 'An Investigation of U.S. Science and Mathematics Education'. The Crossref-registered title for the Springer/Kluwer e-edition is the short form 'A Splintered Vision'. Document analysis plus teacher self-report questionnaires — no student achievement analysis at all. The book PREDATES the TIMSS achievement release it is normally paired with; chapter 6 ('So What Can We Expect from U.S. Students?') is prospective inference from curriculum documents, not an outcome model. No comparison group, no student-level data, no effect estimate anywhere in the volume. This is the origin of the phrase 'a mile wide and an inch deep', which a school founder will be handed as though it were an established causal finding about sequencing. It is a topic count.
Key findings
The US intended curriculum is 'unfocused, repetitive and undemanding' relative to other TIMSS countries. Headline counts: a typical US grade-8 mathematics textbook covers ~35 topics vs ~7 in the typical Japanese equivalent; US grade-4 maths textbooks run 530 pages and science 397 pages against international averages of 170 and 125. Between grades 4 and 8 the US introduces on average ONE topic with focused, in-depth instructional attention; most TIMSS countries introduce 15 and the highest-achieving introduce 20. Only ~25% of grade-8 topics are new since grade 4 in the US, vs ~75% in most TIMSS countries. All of this is a description of BOOKS AND GUIDES. The book contains no test of whether narrowing and re-sequencing the curriculum raises learning, and its authors did not claim to have run one.
Genetic confound
No student outcomes analysed. The causal reading later attached to the book (broad curricula cause low TIMSS scores) is a country-level ecological inference across a handful of nations, confounded with population composition, tracking, private tutoring markets and sampling.
Replication notes
The descriptive claim has been reproduced many times and is not seriously disputed. The causal claim it is used to support has never been tested by any controlled design. Schmidt, Wang & McKnight (2005) extends the same document-analysis method to state standards and explicitly concedes that empirical support for the coherence-causes-learning claim 'must await additional study'.
DOI / URL
10.1007/0-306-47209-0

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Topics per grade-8 mathematics textbook, US vs Japan (descriptive, no outcome measured)topic count~35 (US) vs ~7 (Japan)mixedTIMSS 1995 curriculum corpusnonenot-applicableunclear
Topics given focused in-depth attention between grades 4 and 8topic count1 (US) vs 15 (TIMSS average) vs 20 (highest-achieving countries)mixedTIMSS 1995 curriculum corpusnonenot-applicableunclear

Cited by