The Evidence on Teaching

On the interpretation of below-chance responding in forced-choice tests.

Frederick RI, Speed FM · 2007

grade Dcritiqueindependentnot-applicable
Sample
none - methodological/statistical paper, no sample
Population
Adult forced-choice symptom-validity testing in clinical neuropsychology; the statistical argument is test-agnostic and transfers directly to any multiple-choice subtest.
Design
A statistical exposition, not a study. It is included because it is the only peer-reviewed source that states the rule this archive needs - what a score at or near chance does and does not license - and because it is a rule about arithmetic rather than about a population, so it does not inherit the clinical sample's limits. It carries no effect estimate and no design; grade D is the honest ceiling.
Key findings
Two claims, and the second is the one this archive was missing. (1) A score in the guessing range is uninformative about ability: the examinee's responses carry no signal, so the score is evidence about the administration, not about the person. (2) "Below chance" is a statistical statement with a critical value, not a descriptive one, and clinicians and researchers routinely make "serious errors in communicating what is guessing and what is worse than guessing." A raw score merely under the chance MEAN is not below chance; it must fall outside the sampling distribution of a pure guesser. Most commercial forced-choice tests do not in fact rest their cutoffs on the chance comparison, which is why the distinction gets lost.
Genetic confound
Not applicable - no estimates, no sample, no causal claim.
Replication notes
A statistical argument rather than an empirical finding; replication is not the relevant standard. The binomial result it rests on is not in dispute.

Effects

OutcomeMetricValueMeasureTimingVsHorizonClass
Interpretation of a score inside the guessing rangerulescore carries no information about ability; it is evidence about the administrationstandardizedsingle administrationnonenot-applicabledomain-skill
Threshold for asserting performance is "worse than guessing"rulerequires the score to fall outside the guessing sampling distribution, not merely below its meanstandardizedsingle administrationnonenot-applicabledomain-skill

Cited by