Rates of below-chance performance in forced-choice symptom validity tests.
Greve KW, Binder LM, Bianchini KJ · 2009
grade Clongitudinalindependentreplicated
Sample
1,032 examinees, three symptom-validity tests (PDRT, TOMM, WMT)
Population
Private-practice forensic neuropsychology referrals - alleged mild TBI, moderate-severe TBI, alleged toxic exposure, chronic pain. Adults, not children, and a maximally incentivised sample.
Design
Consecutive-case series reporting base rates, not a designed comparison. Its value here is one number that is otherwise unobtainable: how often a genuinely below-chance result occurs, even in the population most motivated to produce one. Two boundaries on transfer: these are adults with financial incentive to underperform, and the tests are two-alternative forced choice, where chance is 50% and the guessing distribution is far wider relative to the scale than on a four-option achievement subtest.
Key findings
Significantly below-chance performance is RARE even where it is most expected. Rates differ by instrument - PDRT and WMT are equivalent and both yield below-chance results more often than the TOMM - and the harder sections of each test yield more below-chance results than the easier ones, which is what a difficulty-graded guessing model predicts. Using multiple validity tests yields below-chance results more often than any single test, so a single subtest can only ever raise the question, never settle it.
Genetic confound
Not applicable - base rates of a response pattern, no ability comparison and no causal claim about a child.
Replication notes
Three instruments within one sample give an internal cross-check, and the general finding that below-chance responding is rare relative to overall validity-test failure is consistent across the performance-validity literature.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Below-chance performance on forced-choice validity tests | base rate, three instruments | rare in absolute terms; PDRT = WMT > TOMM | standardized | single administration | none | not-applicable | domain-skill |
| Yield of harder vs easier test sections | comparison | seemingly harder sections yield more below-chance results | standardized | single administration | none | not-applicable | domain-skill |
| Single vs multiple validity tests | comparison | multiple tests more likely to detect a below-chance result than one test alone | standardized | single administration | none | not-applicable | domain-skill |
Cited by
- What a standardized achievement score does and does not licensemixedconf: mediumgc: low