The Evidence on Teaching

The implications of symptom validity test failure for ability-based test performance in a pediatric sample.

Kirkwood MW, Yeates KO, Randolph C, Kirk JW · 2012

grade Clongitudinalindependentreplicated
Sample
276 children aged 8-16, consecutive outpatient neuropsychology referrals after mild TBI
Population
US children 8-16 referred after mild traumatic brain injury; overlaps an earlier case series (Kirkwood & Kirk 2010) that also included 17-year-olds.
Design
Consecutive clinical case series with an objective validity test (Medical Symptom Validity Test) administered alongside the ability battery. Not randomised and not a general-population sample - these are clinically referred children after head injury, so the 19% failure rate is a clinical base rate, not a school-testing one. The variance-explained figure is a correlation and does not establish that invalid effort CAUSED the low ability scores rather than sharing a cause with them.
Key findings
The single most useful number in this cluster for an intake tool. Nineteen percent of school-aged children failed the MSVT using adult-derived cutoffs - one child in five, in a sample with no obvious secondary gain. NO background or injury variable distinguished the failers from the passers, so there was no way to predict from the file who would produce an invalid score. Validity-test performance correlated significantly with EVERY ability-based test and explained 38% of total ability-test variance, with large effect sizes across nearly all measures. In other words: whether the child was trying is a bigger determinant of the score profile than most of what the profile is read as measuring.
Genetic confound
Low relevance. The finding is about measurement validity, not about a heritable trait; if anything it cuts against genetic explanation, since the failure was unpredictable from any child or injury characteristic.
Replication notes
Builds on and extends Kirkwood & Kirk (2010) in the same case series; consistent with the wider pediatric performance-validity literature showing children can pass adult-cutoff validity tests, so failure is informative rather than developmental.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Rate of symptom-validity-test failure in school-aged children%19% of 276 children aged 8-16 failed the MSVT actuarial cutoffstandardizedsingle clinical assessmentnonenot-applicabledomain-skill
Share of ability-test variance explained by validity-test performanceR238% of total ability-based test variancestandardizedsingle clinical assessmentnonenot-applicabledomain-skill
Ability scores of validity-test failers vs passersgroup differencesignificantly worse on nearly all neuropsychological tests, large effect sizes across moststandardizedsingle clinical assessmentactive-alternativenot-applicabledomain-skill
Predictors of who produces an invalid scoreassociationno background or injury-related variable differentiated failers from passersstandardizedsingle clinical assessmentnonenot-applicabledomain-skill

Cited by