Exploring Social Metacognition

Lead Research Organisation: University of Oxford
Department Name: Experimental Psychology

Abstract

Summary and relevance:

This project explores the impact of human decision biases in social and group decision making. Using a combination of online and lab-based experiment, agent-based modelling, and social network analysis, the project will explore: a) how people's evaluations of their decisions affects their selection of social partners; b) how biases in this process (such as a preference for others who agree, regardless of accuracy) result in group- and population-level effects like echo chambers; and c) how alternative network structures, information flows, or cognitive strategies might be able to counter these effects.
The project encompasses statistical analysis of psychological experiments, computational modelling of group- and population-level interactions, and analysis of network structures from a variety of real-world datasets. It aims to draw together insights from psychology, sociology, and politics to understand how people in groups make good and bad decisions. This research falls within the MRC portfolio area of "Cognitive and behavioural neuroscience and cognitive systems". It meets the MRC skill priority of Quantitative Skills in its leverage of statistical and computational methods (e.g., agent-based modelling) to explore how biases in individual decision makers can lead to undesirable outcomes on the population level.

Project outline:
People are capable of appraising, remembering, and reasoning about their own thoughts. This capacity for 'metacognition' helps individuals behave adaptively, e.g. knowing how well you know various subject areas can help effective exam preparation. Recent research has suggested that metacognitive information may also be important in social contexts: e.g. group decisions can be optimal if individuals in a group share information about how confident they are in their decisions.

This project builds on recent evidence that people use their metacognitive abilities to evaluate information sources when objective feedback is absent, leading to increased trust in sources which express similar opinions with similar levels of conviction. While adaptive under certain circumstances, this process can amplify biases and may produce echo chambers: ghettoised social groups in which only one set of opinions is represented.
Lab-based experiments will explore the psychology of trust and influence, especially its metacognitive components and the transfer of trust both between individuals and between domains of expertise within a single individual. These experiments will use a judge-advisor paradigm in which individuals faced with a decision receive advice from a range of advisors, combining the advice with their own assessment to reach a final decision on each trial. This paradigm allows for measurement of both trust (how likely a given advisor is to be selected) and influence (how much the advice of a given advisor alters the participant's initial opinion).

Agent-based models and social network analyses of real-world data will explore how the individual-level psychology of trust and influence produces group- and population-level effects such as echo-chambers. These models will additionally be used to suggest methods of reducing the consequences of bias amplification. Online experiments will be used to support and extend findings from both lab-based experiment and agent-based modelling and social network analysis, and to test interventions suggested by modelling.

Publications

10 25 50
 
Title Human online test data 
Description This project contains a great many datasets generated by humans participating in variants of two judge-advisor system tasks. In these tasks participants make a decision, receive some advice pertaining to that decision, and then make a final answer integrating the advice received with the initial decision. 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
Impact These data combine to give us a picture of how humans alter their judgements following advice, in particular the difference made by the availability of objective feedback. 
URL https://osf.io/qtngm/
 
Description RRR Weissman et al 2014 
Organisation Academy of Sciences of the Czech Republic
Country Czech Republic 
Sector Academic/University 
PI Contribution Development of task paradigm in JavaScript. Manuscript editing. Data visualisation and integrity checking.
Collaborator Contribution Project oversight. Task design. Data analysis. Manuscript writing and editing.
Impact Gyurkovics, M., Kovacs, M., Palfi, B., Jaquiery, M., Dechterenko, F., Aczel, B (in press). Registered Replication Report of Weissman, D. H., Jiang, J., & Egner, T. (2014). Determinants of congruency sequence effects without learning and memory confounds. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 2022-2037. Attention, Perception, & Psychophysics.
Start Year 2019
 
Description RRR Weissman et al 2014 
Organisation Eotvos Lorand University
Country Hungary 
Sector Academic/University 
PI Contribution Development of task paradigm in JavaScript. Manuscript editing. Data visualisation and integrity checking.
Collaborator Contribution Project oversight. Task design. Data analysis. Manuscript writing and editing.
Impact Gyurkovics, M., Kovacs, M., Palfi, B., Jaquiery, M., Dechterenko, F., Aczel, B (in press). Registered Replication Report of Weissman, D. H., Jiang, J., & Egner, T. (2014). Determinants of congruency sequence effects without learning and memory confounds. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 2022-2037. Attention, Perception, & Psychophysics.
Start Year 2019
 
Description RRR Weissman et al 2014 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution Development of task paradigm in JavaScript. Manuscript editing. Data visualisation and integrity checking.
Collaborator Contribution Project oversight. Task design. Data analysis. Manuscript writing and editing.
Impact Gyurkovics, M., Kovacs, M., Palfi, B., Jaquiery, M., Dechterenko, F., Aczel, B (in press). Registered Replication Report of Weissman, D. H., Jiang, J., & Egner, T. (2014). Determinants of congruency sequence effects without learning and memory confounds. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 2022-2037. Attention, Perception, & Psychophysics.
Start Year 2019
 
Description RRR Weissman et al 2014 
Organisation University of Sussex
Country United Kingdom 
Sector Academic/University 
PI Contribution Development of task paradigm in JavaScript. Manuscript editing. Data visualisation and integrity checking.
Collaborator Contribution Project oversight. Task design. Data analysis. Manuscript writing and editing.
Impact Gyurkovics, M., Kovacs, M., Palfi, B., Jaquiery, M., Dechterenko, F., Aczel, B (in press). Registered Replication Report of Weissman, D. H., Jiang, J., & Egner, T. (2014). Determinants of congruency sequence effects without learning and memory confounds. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 2022-2037. Attention, Perception, & Psychophysics.
Start Year 2019
 
Description The Confidence Database 
Organisation University Medical Center Hamburg-Eppendorf
Country Germany 
Sector Hospitals 
PI Contribution Submission of datasets and manuscript editing.
Collaborator Contribution Kobe Desender oversaw the collaboration of around 150 confidence researchers submitting databases of confidence studies in humans.
Impact Rahnev, D., Desender, K., Lee, A. L. F., Adler, W. T., Aguilar-Lleyda, D., Akdogan, B., Arbuzova, P., Atlas, L. Y., Balci, F., Bang, J. W., Bègue, I., Birney, D. P., Brady, T. F., Calder-Travis, J., Chetverikov, A., Clark, T. K., Davranche, K., Denison, R. N., Dildine, T. C., Zylberberg, A. (2020). The Confidence Database. Nature Human Behaviour, 1-8. https://doi.org/10.1038/s41562-019-0813-1
Start Year 2019
 
Title Online experiment code 
Description The code in this repository allows the running of the experiments in this project. 
Type Of Technology Webtool/Application 
Year Produced 2018 
Open Source License? Yes  
Impact The code in this project has led to the collection of the various datasets hosted on the OSF (DOI 10.17605/OSF.IO/QTNGM). 
URL https://github.com/oxacclab/ExploringSocialMetacognition