Supporting valid interpretations of learning progression level diagnoses

Secondary-school students responding to ordered multiple-choice items built on the Alonzo & Steedle force-and-motion learning progression; latent class analysis. Exact n not confirmed from the paper.

US secondary physics students; force and motion.

A psychometric validation study of the core assumption behind learning progressions — that a student can be placed at a single level of the progression. Latent class analysis was applied to responses to ordered multiple-choice items keyed to progression levels. The design is not causal and there is no instructional treatment; the study tests whether the CONSTRUCT behind progression-based sequencing survives measurement. Journal of Research in Science Teaching 46(6):699-715. ERIC EJ850695. Read depth is abstract/secondary: the JRST full text is paywalled and the ERIC abstract is truncated mid-sentence, so the specific latent-class solution and sample size are not verified here.

The premise of the whole learning-progression programme is that students consistently express the ideas associated with one level, so that a diagnosis (and hence a sequencing decision) is meaningful. Steedle & Shavelson find this often fails: students respond inconsistently to items about the same idea set in different contexts, and language-interpretation differences further degrade level assignment, so 'valid interpretations of such diagnoses are only possible when students consistently express the ideas associated with a single learning progression level' — a condition their latent class analysis shows is frequently unmet for the force-and-motion progression. This matters for the founder's question directly: if students cannot reliably be located on a progression, a curriculum sequenced to that progression is sequenced to a construct the measurement does not support. This is the closest thing in the literature to a failure-to-validate of a named science learning progression.

Not applicable — measurement study, no outcome comparison.

Consistent with the broader validation record: Alonzo & Steedle (2009, Science Education 10.1002/sce.20303) report the same context-dependence in developing the force-and-motion progression, and later FCI-based attempts to validate force-and-motion progressions (e.g. Singapore secondary schools, IJSME 10.1007/s10763-014-9553-x) report partial fit at best. Harris, Adie & Wyatt-Smith (2022, RER 10.3102/00346543221081552) review learning-progression-based assessment uses and find the evidence base for classroom use thin.