Learning Progressions in Science: An Evidence-Based Approach to Reform

Expert-panel convening plus a narrative review of the (mostly American) science learning-progression literature to 2009. No students, no effect estimates. CPRE Research Report RR-63, Consortium for Policy Research in Education, Teachers College / University of Pennsylvania.

K-12 science learning progressions in the US research literature.

Grey literature: a funded expert-panel report, not a study. Its value to this archive is that it is the field's own most careful audit of how much empirical backing learning progressions actually had at the moment they were about to be built into national policy (the NRC Framework followed in 2012 and NGSS in 2013). Read in full text.

The report defines learning progressions as 'empirically grounded and testable hypotheses' about how understanding develops, and distinguishes two validation questions: construct validity (does the hypothesised sequence describe the path most students actually take?) and CONSEQUENTIAL validity (does instruction based on the progression produce better results?). On construct validity it finds the field 'at the initial stages of producing evidence on the degree of validity and usefulness of these progressions', not yet covering enough content 'for them to be able to bear the full weight of replacing standards'. On consequential validity — the question a school founder actually cares about — its verdict is blunt: 'At this point, this is among the most hypothetical of questions.' It then sketches the study that would be needed (states or districts re-array standards to match a progression; curriculum and PD keyed to it; the whole regime evaluated against business-as-usual) and calls for funding an 'existence proof' that a well-tested progression improves outcomes. As of 2009, no such demonstration existed. The report also explicitly rejects the premise that there is one right order: 'it is not assumed that there is likely to be one best progression or pathway. The number and nature of productive pathways are treated as empirical questions.'

Not applicable — no outcome analysis. Note that the descriptive evidence base underneath progressions (cross-sectional studies of what students of different ages believe) is exactly the kind of observational work this archive treats as presumptively confounded by maturation and selection.

The consequential-validity gap the report identifies in 2009 was still open when NGSS was adopted in 2013 and remains open: no randomized or well-identified quasi-experiment has shown that instruction sequenced to a validated learning progression outperforms instruction that is not. Attempts at construct validation have themselves run into trouble (see Steedle & Shavelson 2009).