Finding time for computer science in the elementary school day: a quasi-experimental study of a transdisciplinary problem-based learning approach
Century, J., Ferris, K. A., & Zuo, H. · 2020
grade Cquasi-experimentdeveloper-ledunreplicated
Sample
16 elementary schools, 321 teachers, 5,791 students in grades 3-5, Broward County Public Schools
Population
Grades 3-5 in one very large, diverse US district.
Design
Schools were matched a priori on six characteristics and then RANDOMLY assigned to treatment or comparison — a randomised block design at school level, stronger than the quasi-experimental label suggests. But the analysis is not intention-to-treat: the reported results are associations between implementation dose (the percentage of Code.org lessons completed) and achievement, which reintroduces exactly the selection the randomisation removed. And there is no programming-skill outcome at all; the outcomes are Florida state English and mathematics scores and attitudes toward computer science.
Key findings
Teachers who taught the Time4CS modules completed more Code.org lessons than teachers who did not. Completing a higher percentage of non-grade-level Code.org Fundamentals lessons was positively ASSOCIATED with state English and mathematics scores, and more interdisciplinary teaching was associated with higher English scores. Every headline claim is a dose-response correlation inside a randomised design that could have supported a clean intention-to-treat estimate.
Genetic confound
Medium-to-high for the reported estimates. The randomisation is discarded in favour of a dose-response analysis, and teachers who complete more lessons are not a random subset.
Replication notes
No replication; a single-district study.
DOI / URL
10.1186/s40594-020-00218-3
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Code.org lesson completion (implementation fidelity) | group contrast | treatment teachers completed more lessons | administrative | end of school year | business-as-usual | end-of-treatment | behaviour |
| State English and mathematics achievement | association with lesson dose | positive association — not an intention-to-treat effect | standardized | end of school year | unclear | end-of-treatment | far-transfer |