Measuring Critical Thinking: Results From an Art Museum Field Trip Experiment

67 matched pairs of applicant school groups completed the lotteries (134 groups), >7,500 students across 111 schools; analysed n = 3,610 (representational art) and n = 3,958 (abstract art)

Grades 3-12 school groups applying for guided tours of Crystal Bridges Museum of American Art, Arkansas; surveys administered a mean of 21.8 days after the treatment group's visit

A lottery among oversubscribed applicants, which is the identification strategy this archive treats most favourably - assignment is exogenous and the applicant pool is self-selected in both arms equally. The limit is the outcome: students wrote an essay about an unfamiliar painting, blind-coded on a seven-item critical-thinking rubric built by the research team. There is no administrative or standardised achievement measure anywhere in the study, so this is evidence about looking at art, not about school achievement. The research team publishes a programme of positive arts-field-trip findings, hence developer-involved.

Treatment students scored 0.08-0.13 SD higher on blind-rated critical analysis of an unfamiliar painting - representational art 0.10 (SE 0.03-0.05, n = 3,610), abstract art 0.13 (SE 0.05) parsimonious and 0.08 (SE 0.04) with covariates (n = 3,958). Effects were larger for students from higher free-lunch schools, significant in the representational sample and just short of significance in the abstract sample. The honest reading of the effect is near transfer: an hour guided through paintings makes students better at writing about a painting three weeks later. The earlier Education Next report of the same programme (10,912 students) adds historical empathy +0.06, tolerance +0.07 and future art-museum interest +0.08, all researcher-made measures with standard errors not published.

Low. Lottery assignment within an applicant pool means both arms are equally self-selected; family and genetic differences cannot differ systematically between arms.

The core result reproduces across two independent waves within the programme (representational art in spring, abstract art in autumn), on different paintings and different students, which is why replication is recorded as replicated. It has not been reproduced by an independent research group, and the outcome instrument is the team's own. Both waves are also included inside the AIR ESSA review, where they supply the entire critical-thinking cell - so the two sources are not independent of one another.