The Evidence on Teaching

Success in introductory college physics: The role of high school preparation

Sadler, P. M., & Tai, R. H. · 2001

grade Dlongitudinalindependentreplicatednumbers spot-checked
Sample
1,933 students in 19 introductory physics courses at 18 US colleges and universities (9 public, 8 private, 1 military academy; 9 calculus-based, 10 algebra-based); surveyed fall 1994.
Population
US undergraduates in first-semester introductory college physics, reporting retrospectively on their high school physics and mathematics.
Design
Retrospective self-report survey (57 items) with professor-supplied grades on a 100-point scale; reverse-stepwise multiple regression with each college as a dummy. Recruitment was professor-discretionary — 100 randomly selected physics professors were contacted, 38 taught the course, and 19 courses ended up in the sample — and the authors concede participation 'may have been a function of the satisfaction of the professor with his or her own teaching'. About 25% of AP students place out of introductory physics and are therefore MISSING from the dataset (a robustness check drops AP students). All exposure variables are student recall of courses taken years earlier; an anonymous reviewer's objection is reproduced verbatim in the paper's own footnote: 'there are many uncontrolled variables, and the reliance of students to accurately recall and report on their physics experiences is a serious weakness'. Read in full text.
Key findings
Taking high school physics is associated with a college physics grade about 2.3 points higher on 100 (raw gap 82.1 vs 79.8, d = 0.24) — and taking high school CALCULUS is worth about the same (+2.58) as a year of regular physics (+2.26). Honors physics +3.51, AP physics +4.32, a second year of physics +2.80. Explicitly NOT significant: whether the student took high school chemistry or biology — i.e. the cross-discipline prerequisite claim fails here too. The finding most useful for sequencing decisions is Model B, restricted to students who did take high school physics: once teaching practices enter the model, the LEVEL of the course (regular/honors/AP) stops being significant, and COVERAGE carries a negative coefficient (−0.49). The abstract states it directly: 'Students who had high school courses that spent more time on fewer topics, concepts, problems, and labs performed much better in college than those who raced through more content in a textbook-centered course.' Note also a calibration point the archive should keep: students' own estimate of how much high school physics helped them averaged +7.8 points, about THREE TIMES the measured association — and students who had never taken it rated it more valuable than those who had.
Genetic confound
HIGH, and uniquely well-documented by the authors themselves. From the abstract: 'Although students without a high school physics course often do well in college physics, they are more likely to be academically stronger, with more educated parents, having previously taken calculus, and taking physics in their sophomore or junior year in college.' The authors' own epistemic statement is the one to quote: 'the methodology for this study is considered epidemiological rather than experimental... We understand that our study cannot claim causal connections between variables and outcomes, but it can identify key relationships that are worthy of controlled studies.' They add the asymmetry that makes the NULL results the more useful half: 'While a high correlation is not evidence for causality, a low or negative correlation is evidence against a positive causal relationship.' That licenses the cross-discipline null (chemistry and biology do not predict college physics) far more than it licenses the positive coefficients.
Replication notes
Same-programme replication across the FICSS family: Tai, Sadler & Loehr 2005 (chemistry), Sadler & Tai 2007 (all three disciplines), Schwartz et al. 2009 (depth vs breadth, n = 8,310 at 55 institutions), where the depth finding reappears with full controls. No independent replication on a different dataset; no causal design has ever tested the depth-over-breadth claim.
DOI / URL
10.1002/1098-237X(200103)85:2<111::AID-SCE20>3.0.CO;2-O

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
College physics grade, took high school physics vs not (raw)d0.24teacher-assignedend of first college physics coursenoneover-2yrdomain-skill
College physics grade, regular high school physics (adjusted)grade points (100-scale)+2.26teacher-assignedend of first college physics coursenoneover-2yrdomain-skill
College physics grade, took high school calculusgrade points (100-scale)+2.58teacher-assignedend of first college physics coursenoneover-2yrnear-transfer
College physics grade, took high school chemistry or biology (cross-discipline prerequisite)grade points (100-scale)not significantteacher-assignedend of first college physics coursenoneover-2yrnear-transfer
College physics grade, per unit of high school topic COVERAGE (breadth), among physics-takersregression coefficient-0.49 (more topics covered, worse college grade)teacher-assignedend of first college physics coursenoneover-2yrdomain-skill

Cited by