A method for analyzing the coherence of high school biology textbooks

4 widely used US high school biology textbooks; one topic (matter and energy transformations in living systems); trained reviewers, no students.

US high school biology textbooks. No learners appear in the study.

AAAS Project 2061 instrument-development study. Trained reviewers applied a rubric derived from the Atlas of Science Literacy strand maps to judge whether four textbooks presented a coherent account of one topic. The measured outcome is INTER-REVIEWER CONSISTENCY of the rubric, not student learning. There is no comparison group, no achievement measure and no effect on anything a student does. Print issue is JRST 47(1):47-70, January 2010; Crossref's 2009-12-04 date is EarlyView. The authors themselves position the work as pre-empirical: the Project 2061 approach 'can be a cost-effective first step towards identifying materials that are ready for large-scale and more expensive empirical testing'.

A reliable rubric can be built for judging textbook coherence, and the four popular high school biology textbooks reviewed did NOT provide a coherent account of matter and energy transformations — they failed on interrelated age-appropriate ideas with explicit connections, effective representations, use of the ideas to explain phenomena, and avoidance of distracting technical detail. The study establishes that coherence can be RATED; it establishes nothing about whether a more coherent text produces more learning. Recorded as excluded because it contributes no evidence to the causal question the topic asks — but it is the methodological backbone of the coherence literature and a founder will meet its ratings in curriculum-review products (EdReports, NextGenScience) presented as if they were efficacy evidence.

No student outcomes; not applicable.

The rubric approach has been extended by AAAS and by EdReports/NextGenScience curriculum reviews. Its predictive validity for learning has not been established by any controlled study. The one comparison-group test from the same team (Herrmann-Abell, Koppal & Roseman 2016, CBE-LSE 10.1187/cbe.16-03-0112, 6 grade-8 teachers, ~600 students, randomized) is developer-led and small.