Signal detection theory can be applied to single-stimulus designs by using confidence ratings
the verdict
SUPPORTED
the evidence backs this
refutedsupported
the weight of evidence
6 sources for · 0 against
Peer-reviewed literature demonstrates that signal detection theory models can be applied to perceptual and stimulus detection paradigms by using confidence ratings and category thresholds.
Confidence judgements, self-assessments about the quality of a subject's knowledge, are considered a central example of metacognition. Prima facie, introspection and self-report appear the only way to access the subjective sense of confidence or uncertainty. Contrary to this notion, overt behavioural measures can be used to study confidence judgements by animals trained in decision-making tasks with perceptual or mnemonic uncertainty. Here, we suggest that a computational approach can clarify the issues involved in interpreting these tasks and provide a much needed springboard for advancing the scientific understanding of confidence. We first review relevant theories of probabilistic inference and decision-making. We then critically discuss behavioural tasks employed to measure confidence in animals and show how quantitative models can help to constrain the computational strategies underlying confidence-reporting behaviours. In our view, post-decision wagering tasks with continuous measures of confidence appear to offer the best available metrics of confidence. Since behavioural reports alone provide a limited window into mechanism, we argue that progress calls for measuring the neural representations and identifying the computations underlying confidence reports. We present a case study using such a computational approach to study the neural correlates of decision confidence in rats. This work shows that confidence assessments may be considered higher order, but can be generated using elementary neural computations that are available to a wide range of species. Finally, we discuss the relationship of confidence judgements to the wider behavioural uses of confidence and uncertainty.
Signal detection theory (SDT) is used to quantify people’s ability and bias in discriminating stimuli. The ability to detect a stimulus is often measured through confidence ratings. In SDT models, the use of confidence ratings necessitates the estimation of confidence category thresholds, a requirement that can easily result in models that are overly complex. As a parsimonious alternative, we propose a threshold SDT model that estimates these category thresholds using only two parameters. We fit the model to data from Pratte et al. (Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 224–232 2010) and illustrate its benefits over previous threshold SDT models.
Summary This study examines signal detection theory (SDT) analysis of perceptual detection performance using response time (RT) data. A defining feature of detection tasks is the asymmetry between trials with stimulus presence and absence, often reflected in asymmetric type-1 ROC curves. This asymmetry indicates greater signal variability in stimulus-present trials, which contradicts canonical assumptions in equal-variance SDT models. Across multiple datasets, we implemented an unequal-variance SDT model using RT data and compared it with the traditional confidence-based method. RT-based estimates of SDT parameters—SD ratio (σ) and mean difference (μ)—aligned closely with confidence-based estimates. The resulting sensitivity measure, da—an unequal-variance extension of d′—derived from RT and confidence, showed strong consistency. Notably, conventional d′ systematically overestimated detection performance compared to the da measures, highlighting the importance of accounting for unequal variance. RT-based SDT analysis offers a cost-effective alternative for robustly quantifying detection performance, particularly when confidence ratings are impractical.
Signal detection theory (SDT) has long provided the field of psychology with a simple but powerful model of how observers make decisions under uncertainty. SDT can distinguish sensitivity from response bias and characterize optimal decision strategies. Whereas classical SDT pertains to "type 1" judgments about the world, recent work has extended SDT to quantify sensitivity for metacognitive or "type 2" judgments about one's own type 1 processing, e.g. confidence ratings. Here we further advance the application of SDT to the study of metacognition by providing a formal account of normative metacognitive decision strategies - i.e., type 2 (confidence) criterion setting - for ideal observers. Optimality is always defined relative to a given objective. We use SDT to derive formulae for optimal type 2 criteria under four distinct objectives: maximizing type 2 accuracy, maximizing type 2 reward, calibrating confidence to accuracy, and maximizing the difference between type 2 hit rate and false alarm rate. Where applicable, we consider these optimization contexts alongside their type 1 counterparts (e.g. maximizing type 1 accuracy) to deepen understanding. We examine the different strategies implied by these formulae and further consider how optimal type 2 criterion setting differs when metacognitive sensitivity deviates from SDT expectation. The theoretical framework provided here can be used to better understand the metacognitive decision strategies of real observers. Possible applications include characterizing observers' spontaneously chosen metacognitive decision strategies, assessing their ability to fine-tune metacognitive decision strategies to optimize a given outcome when instructed, determining over- or under-confidence relative to an optimal standard, and more. This framework opens new avenues for enriching our understanding of metacognition.
Perception is biased by expectations and previous actions. Pre-stimulus brain oscillations are a potential candidate for implementing biases in the brain. In two EEG studies (43 and 39 participants) on somatosensory near-threshold detection, we investigated the pre-stimulus neural correlates of an (implicit) previous choice bias and an explicit bias. The explicit bias was introduced by informing participants about stimulus probability on a single-trial level (volatile context) or block-wise (stable context). Behavioural analysis confirmed adjustments in the decision criterion and confidence ratings according to the cued probabilities and previous choice-induced biases. Pre-stimulus beta power with distinct sources in sensory and higher-order cortical areas predicted explicit and implicit biases, respectively, on a single subject level and partially mediated the impact of previous choice and stimulus probability on the detection response. We suggest pre-stimulus beta oscillations in distinct brain areas as a neural correlate of explicit and implicit biases in somatosensory perception.
The relationship between accuracy and confidence in psychophysical tasks traditionally has been assumed to be mainly positive, i.e., the two typically increase or decrease together. However, recent studies have reported examples of exceptions, where confidence and accuracy dissociate from each other. Explanations for such dissociations often involve dual-channel models, in which a cortical channel contributes to both accuracy and confidence, whereas a subcortical channel only contributes to accuracy. Here, we show that a single-channel model derived from signal detection theory (SDT) can also account for such dissociations. We applied transcranial magnetic stimulation (TMS) to the occipital cortex to disrupt the internal representation of a visual stimulus. The results showed that consistent with previous research, occipital TMS decreased accuracy. However, counterintuitively, it also led to an increase in confidence ratings. The data were predicted well by a single-channel SDT model, which posits that occipital TMS increased the variance of the internal stimulus distributions. A formal model comparison analysis that used information theoretic methods confirmed that this model was preferred over single-channel models, in which occipital TMS changed the signal strength or dual-channel models, which assume two different processing routes. Thus our results show that dissociations between accuracy and confidence can, at least in some cases, be accounted for by a single-channel mo
Everything we examined (6)
This check searched the claim as stated. It did not run a separate search for evidence against it.