On the interpretation of below-chance responding in forced-choice tests.
Frederick RI, Speed FM · 2007
grade Dcritiqueindependentnot-applicable
Sample
none - methodological/statistical paper, no sample
Population
Adult forced-choice symptom-validity testing in clinical neuropsychology; the statistical argument is test-agnostic and transfers directly to any multiple-choice subtest.
Design
A statistical exposition, not a study. It is included because it is the only peer-reviewed source that states the rule this archive needs - what a score at or near chance does and does not license - and because it is a rule about arithmetic rather than about a population, so it does not inherit the clinical sample's limits. It carries no effect estimate and no design; grade D is the honest ceiling.
Key findings
Two claims, and the second is the one this archive was missing. (1) A score in the guessing range is uninformative about ability: the examinee's responses carry no signal, so the score is evidence about the administration, not about the person. (2) "Below chance" is a statistical statement with a critical value, not a descriptive one, and clinicians and researchers routinely make "serious errors in communicating what is guessing and what is worse than guessing." A raw score merely under the chance MEAN is not below chance; it must fall outside the sampling distribution of a pure guesser. Most commercial forced-choice tests do not in fact rest their cutoffs on the chance comparison, which is why the distinction gets lost.
Genetic confound
Not applicable - no estimates, no sample, no causal claim.
Replication notes
A statistical argument rather than an empirical finding; replication is not the relevant standard. The binomial result it rests on is not in dispute.
Effects
| Outcome | Metric | Value | Measure | Timing | Vs | Horizon | Class |
|---|---|---|---|---|---|---|---|
| Interpretation of a score inside the guessing range | rule | score carries no information about ability; it is evidence about the administration | standardized | single administration | none | not-applicable | domain-skill |
| Threshold for asserting performance is "worse than guessing" | rule | requires the score to fall outside the guessing sampling distribution, not merely below its mean | standardized | single administration | none | not-applicable | domain-skill |
Cited by
- What a standardized achievement score does and does not licensemixedconf: mediumgc: low