Impact of project-based curriculum materials on student learning in science: Results of a randomized controlled trial

42 schools randomised (21 treatment, 21 comparison); 94 teachers contributed data (55 treatment, 39 comparison); more than 2,400 grade-6 students

Grade 6 science, one large urban US school district.

All middle schools in the district were randomly assigned; treatment teachers taught Project-Based Inquiry Science (PBIS) units, comparison teachers taught the district-adopted textbook (McDougal Littell) on the SAME two topics (an energy/physical science unit and an Earth science unit). Teachers in BOTH arms received professional development on the NRC Framework, which is unusually clean. PBIS is heavily structured project-based inquiry — an 8-10 week unit framed by a "big question", with scripted sub-questions, a class project board, and small-group investigations — not unassisted discovery. The two outcome measures were built by the research team to assess Framework-aligned practices (constructing explanations; developing and using models) and scored by trained raters. TWO INDEPENDENCE FLAGS. First, Krajcik is a PBIS/LeTUS developer and is on the author list (SRI led the evaluation, hence developer-involved rather than developer-led). Second, the What Works Clearinghouse reviewed this trial in its March 2020 Project-Based Inquiry Science intervention report and ruled that it "does not meet WWC group design standards because equivalence of the analytic intervention and comparison groups is necessary and not demonstrated" — 85% of comparison teachers versus 96% of treatment teachers contributed data. WWC found that NO study of PBIS (seven reviewed) meets its standards, so it can draw no conclusion about the programme at all.

On researcher-built, Framework-aligned end-of-unit tests: physical science (energy) unit d=+0.21, p=0.04; Earth science unit d=+0.25, p=0.056 (i.e. not significant at 0.05). Baseline differences favoured the COMPARISON group (d=+0.09 in science to +0.22 in reading), so the estimate is conservative in that respect, but WWC still rejected the analytic sample for non-equivalence. The report's own literature comparison is the most useful sentence in it for this archive: past PBIS evaluations produced "moderate (+0.37 to +0.44) for larger studies that relied on standardized test scores to large (+1.0 to +1.5) for smaller-scale studies that relied on researcher-constructed performance tasks closely aligned to the learning goals of the materials" — a roughly 3x measure-alignment gradient stated by the programme's own evaluators.

Low as designed (school-level randomisation within one district), but the WWC finding of non-equivalence in the analytic sample means differential teacher non-participation, not heredity, is the live threat.

The only other large PBIS evaluation is Geier 2008 (Detroit), also rejected by WWC for non-equivalence. WWC March 2020 concludes no PBIS study meets its standards. Treat the PBIS evidence base as unreplicated and unverified rather than positive.