The mind as a sampler: Towards a standard model for cognitive science
Lead Research Organisation:
University of Warwick
Abstract
Standard models are scientific crown jewels, synthesising and organising existing knowledge to make empirically-testable predictions in new domains. Probabilistic rational models, implemented as either Bayesian models or deep neural networks, have been proposed as standard models, from low-level perception and neuroscience to cognition and economics. But capturing real human mental performance, constrained by limited time and subject to systematic errors, requires transforming rational models into process models that approximate probabilistic calculations in real time in the human brain.
This presents a key research challenge: is there a “standard process model” for probabilistic rational theories of cognition? That is, can we create a standard, systematic way to turn pure rational probabilistic models into quantitative process models that capture a wide range of behavioural measures?
This project builds on the recently-developed Autocorrelated Bayesian Sampler (ABS; Zhu, et al., 2024): a task-independent process that samples from a probabilistic model and uses proven mechanisms to produce decisions, response times, confidence judgments, and other response modes. With the ABS, the underlying probabilistic rational model shapes behaviour around the constraints of the task. If the sampling process continued indefinitely, the ABS would be a pure rational model; but, crucially, psychologically-realistic sample sizes and autocorrelated samples produce biases and trial-to-trial variability.
Empirically, this project prepares the ground for a standard model by first establishing the generality of key biases across tasks and across the perceptual-cognitive divide. Building upon our previous work (Spicer, et al., 2022), cognitive biases such as anchoring and over-precision will be investigated in several perceptual tasks, including numerosity, motion perception, and ensemble prediction. Conversely, perceptual biases, such as repulsion from a decision bound, will be investigated in several cognitive tasks, including iterative prediction, answering general knowledge questions, and numeric ensembles. We will also run additional experiments using the whole battery of perceptual and cognitive tasks, to assess co-occurrence and individual differences in biases.
The ABS is a state-of-the-art process model in iterative forecasting and probability judgement (Spicer, et al., in press; Sundh, et al., 2023). But in other domains it has only been tested qualitatively. Therefore, the first computational modelling objective is to quantitatively compare the ABS against leading models of perception and cognition in this wider set of tasks. In perception, this will involve quantitatively comparing the ABS to standard probabilistic rational models and existing process models (e.g., drift diffusion models) to determine if it accurately captures biases as well as the performance deficit participants show relative to Bayesian models. In cognition, this will involve quantitatively comparing the ABS to existing sampling and heuristic models. We will then assess whether the same computational motifs, such as small sample sizes and the degree of sample autocorrelation, hold for all tasks, across the perceptual-cognitive divide. The second computational objective is to use the ABS parameters fit in one task to predict responses in other tasks to determine whether a consistent process underlies each task and identify new precise individual differences measures (e.g., sampling rate).
Overall, this project aims to produce high-profile papers and software packages highlighting the within- and cross-domain successes of the model. These will provide critical momentum toward a standard method for creating detailed process models that predict reaction times, variability, biases and so on. This builds on existing probabilistic rational models, and will build on new AI models as they develop.
This presents a key research challenge: is there a “standard process model” for probabilistic rational theories of cognition? That is, can we create a standard, systematic way to turn pure rational probabilistic models into quantitative process models that capture a wide range of behavioural measures?
This project builds on the recently-developed Autocorrelated Bayesian Sampler (ABS; Zhu, et al., 2024): a task-independent process that samples from a probabilistic model and uses proven mechanisms to produce decisions, response times, confidence judgments, and other response modes. With the ABS, the underlying probabilistic rational model shapes behaviour around the constraints of the task. If the sampling process continued indefinitely, the ABS would be a pure rational model; but, crucially, psychologically-realistic sample sizes and autocorrelated samples produce biases and trial-to-trial variability.
Empirically, this project prepares the ground for a standard model by first establishing the generality of key biases across tasks and across the perceptual-cognitive divide. Building upon our previous work (Spicer, et al., 2022), cognitive biases such as anchoring and over-precision will be investigated in several perceptual tasks, including numerosity, motion perception, and ensemble prediction. Conversely, perceptual biases, such as repulsion from a decision bound, will be investigated in several cognitive tasks, including iterative prediction, answering general knowledge questions, and numeric ensembles. We will also run additional experiments using the whole battery of perceptual and cognitive tasks, to assess co-occurrence and individual differences in biases.
The ABS is a state-of-the-art process model in iterative forecasting and probability judgement (Spicer, et al., in press; Sundh, et al., 2023). But in other domains it has only been tested qualitatively. Therefore, the first computational modelling objective is to quantitatively compare the ABS against leading models of perception and cognition in this wider set of tasks. In perception, this will involve quantitatively comparing the ABS to standard probabilistic rational models and existing process models (e.g., drift diffusion models) to determine if it accurately captures biases as well as the performance deficit participants show relative to Bayesian models. In cognition, this will involve quantitatively comparing the ABS to existing sampling and heuristic models. We will then assess whether the same computational motifs, such as small sample sizes and the degree of sample autocorrelation, hold for all tasks, across the perceptual-cognitive divide. The second computational objective is to use the ABS parameters fit in one task to predict responses in other tasks to determine whether a consistent process underlies each task and identify new precise individual differences measures (e.g., sampling rate).
Overall, this project aims to produce high-profile papers and software packages highlighting the within- and cross-domain successes of the model. These will provide critical momentum toward a standard method for creating detailed process models that predict reaction times, variability, biases and so on. This builds on existing probabilistic rational models, and will build on new AI models as they develop.