Investigating New Ways to Improve Eyewitness Identifications Using Receiver Operating Characteristic Analysis

Lead Research Organisation: Royal Holloway, University of London
Department Name: Psychology

Abstract

There is a two-pronged, very real societal problem concerning identifications made by eyewitnesses: innocent suspects are mistakenly identified and charged with a crime they did not commit, or guilty suspects are not identified and free to commit more crimes. Decreasing the chances that innocent suspects are misidentified, unfortunately also decreases the chances that guilty suspects are identified; and likewise, increasing the chances that guilty suspects are identified also increases the chances that innocent suspects are identified. In other words, an eyewitness's accuracy is not just about choosing the right suspect; it is also about not misidentifying the wrong suspect. This teeter tottering is what eyewitness memory researchers have been grappling with for decades, and we aim to combat it.

That may seem like a bold claim, but it is now possible because of the new method of analysis we recently introduced to the field of eyewitness memory. The method disentangles the concept of response bias (i.e., the inclination of an eyewitness to choose someone or not choose someone from an identification parade) from discriminability (i.e., the ability to discriminate an innocent suspect from a guilty suspect). While the method is new to eyewitness memory researchers, the method is tried-and-true for medical diagnosticians who routinely use it to test whether one diagnostic procedure is better able to discriminate a disease state from a non-disease state, for example. Eyewitness memory researchers are faced with the same conceptual issue as medical diagnosticians in which they need to determine whether one procedure is better able to discriminate guilty from innocent suspects. Our new method allows us to answer important questions in eyewitness memory that could not be previously answered because there were no satisfactory ways to measure discriminability.

We will use our new technique in a series of experiments designed to investigate ways to increase discriminability. We will do this by comparing commonly used identification parade procedures and manipulating components of each procedure to investigate the factors that increase discriminability. The results of the experiments will shed light on why certain identification parade procedures (e.g., sequentially presented videos vs. simultaneously presented videos) and components of those procedures (e.g., identifications made quickly, number of individuals presented in an identification parade) increase discriminability. The results of this research could ultimately lead to more guilty people and fewer innocent people being identified and later prosecuted. Thus, the proposed research has the potential to make significant societal impact.

Planned Impact

The appeal of this work is wide ranging both in terms of applying a new methodology in the field of eyewitness research as well as advancing research based practice when it comes to identification parades.

By identifying factors that increase the accuracy of identifications, we expect the following individuals and groups to benefit:

Professionals in the Criminal Justice System, including:
- Police Investigators
- Solicitors
- Policymakers
Advocacy Groups, including:
- Victim Support Advocates
- The Innocence Network
Members of Society, including:
- Victims
- Innocent suspects
- Eyewitnesses

Communication with user groups is critically important for this type of research because if researchers and practitioners do not communicate, our respective efforts will fall on deaf ears. We have begun to and will continue to liaise with Metropolitan (MET) Police Inspector Burnell. Inspector Burnell is in charge of the MET Identification Suites. He is a critical contact person for sharing information, as well as providing the resources necessary to carry out the proposed experiments. We will continue to work closely together to maximise impact (please see his accompanying letter of support).

We will build a network of experts from various professions (i.e., stakeholders from the list above) to disseminate the results and receive feedback. As we disseminate the results and attend conferences and research fairs, the network will grow. These individuals report to the Association of Chief Police Officers and the Crown Prosecution Services committees, so this will be a step towards expanding the network to include more officers and solicitors. If the findings from the experiments yield results that could improve PACE Code guidelines, then a next logical step would be to lobby the Home Office to make procedural changes.

We will also liaise with members of victims advocacy groups of two varieties: one for victims of crimes and the other for victims of the criminal justice system (i.e., innocent suspects who are wrongly convicted). There is a legacy of some researchers who only liaise with groups who support innocent suspects and their work comes across as one-sided. This is unfortunate because there are trade-offs associated with eyewitness identifications (e.g., fewer innocent suspects identified lead to fewer guilty suspects identified; Clark, 2012).

A workshop is planned for the end of the project to disseminate the findings of the research to those who aim to make the criminal justice system more efficient, including identification officers, solicitors, victim rights advocates, and policymakers. The workshop will allow for us to receive valuable feedback. We will also create and maintain an online forum for the network to regularly communicate.

To inform the general public, we will promote the work through contact with the media and participation in public science events. We will also build and maintain a web presence dedicated to providing updates of the research for the user groups and general public (e.g., our laboratory websites and social media accounts). We hope this will invite researchers, students, and the general public to be informed by and comment on our research.

Publications

10 25 50
publication icon
Mickes L (2017) Eyewitness Identification

publication icon
Mickes L (2016) The effects of verbal descriptions on eyewitness memory: Implications for the real-world in Journal of Applied Research in Memory and Cognition

publication icon
Mickes L (2017) ROCs in Eyewitness Identification: Instructions versus Confidence Ratings in Applied Cognitive Psychology

publication icon
Mickes L (2017) Distilling the Confidence-Accuracy Message: A Comment on Wixted and Wells (2017). in Psychological science in the public interest : a journal of the American Psychological Society

publication icon
Mickes Laura (2016) The Effects of Verbal Descriptions on Eyewitness Memory: Implications for the Real-World in JOURNAL OF APPLIED RESEARCH IN MEMORY AND COGNITION

publication icon
Seale-Carlisle TM (2016) US line-ups outperform UK line-ups. in Royal Society open science

publication icon
Wilson BM (2018) The effects of verbal descriptions on performance in lineups and showups. in Journal of experimental psychology. General

publication icon
Wixted J (2015) ROC analysis measures objective discriminability for any eyewitness identification procedure in Journal of Applied Research in Memory and Cognition

publication icon
Wixted J (2015) Evaluating eyewitness identification procedures: ROC analysis and its misconceptions in Journal of Applied Research in Memory and Cognition

publication icon
Wixted John T. (2017) ROC Analysis in Theory and Practice in JOURNAL OF APPLIED RESEARCH IN MEMORY AND COGNITION

 
Description Our findings, thus far, are:

1. when lineup members are shown at the same time accuracy is better than when the lineup members are presented sequentially (as is current practice in the UK)
2. lapping twice (as is current practice in the UK) vs. once through the lineup does not make a significant difference.
3. videos (as is current practice in the UK) and photo lineups yield the same accuracy
4. US lineups yield better accuracy than UK lineups
5. found further evidence that confidence in an identification is informative of accuracy
Exploitation Route If the results replicate then policymakers may recommend:

- changing lineup presentation from sequential to simultaneous to increase the likelihood that the guilty suspect is identified and the innocent suspect is not identified
- creating videos (that use digital resources) are not necessary when photos are sufficient
- lapping through the lineup twice is unnecessary
- confidence needs to be collected at initial identification
Sectors Government, Democracy and Justice

URL http://www.mickeslab.com/
 
Description European Association of Psychology and Law Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Postdoctoral research assistant presented work.

Wetmore, S. A., Mickes, L., Seale-Carlisle, T. M., Morgan, D. P., Clark, S., Gronlund, S. D., & Wixted, J. T. More than one kind of ROC: Biasing ROCs and confidence-based ROCs in eyewitness identification. European Association of Psychology and Law, Nuremberg, Germany, August 2015.
Year(s) Of Engagement Activity 2015
 
Description European Association of Psychology and Law Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Presented results.

Mickes, L. Results from Certain Analyses Matter to Certain Decision-Makers. Annual European Association of Psychology and Law, Nuremberg, Germany, August 2015.
Year(s) Of Engagement Activity 2015
 
Description Exploring Psychology at Royal Holloway 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Approximately 200 A-level students attended the RHUL Psychology Taster day. We ran two sessions where students took part in a real experiment and were given a presentation about the (un)reliability of eyewitness identifications.
Year(s) Of Engagement Activity 2016
URL https://www.royalholloway.ac.uk/studyhere/educationaladvisorsandschools/activitiesandevents/tasterco...
 
Description Hosted and presented roundtable event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Collecting and recording eyewitnesses' confidence was the message we conveyed to stakeholders at a roundtable event on the 8th of December 2016. The stakeholders included the Director of the Police Foundation (who chaired the roundtable), a publisher from a Police Trade Magazine (Policing Insight), two former police officers, one current police officer who specialises in IDs, IT Director of VIPER system for creating IDs, two individuals from the Major's Office of Policing and Crime, a senior scientist at the Home Office, one Police and Crime Commissioner, etc.
Year(s) Of Engagement Activity 2016
URL http://www.mickeslab.com/?page_id=941
 
Description International Conference on Memory Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Over 1000 memory researchers attended this conference. I presented the following paper:

Mickes, L., Clark, S. E., Gronlund, S. D., Roediger, H. L., & Wixted, J. T. (2016). Eyewitnesses: Not so unreliable after all. International Conference of Memory, Budapest, Hungary, July 2016.
Year(s) Of Engagement Activity 2016
URL http://www.icom2016.com
 
Description International Conference on Memory Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The postdoctoral research assistant presented work:

Wetmore, S. A., Flowe, H. D., Wixted, J. T., & Mickes, L. (2016). UK lineups should be simultaneously presented. International Conference of Memory, Budapest, Hungary, July 2016.
Year(s) Of Engagement Activity 2016
URL http://www.icom2016.com
 
Description International Meeting of the Psychonomic Society 2016 Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The postdoctoral research assistant on the grant presented findings.

Wetmore, S. A., Mickes, L., & Wixted, J. T. Using Lineup Instructions to Manipulate Response Bias and its Relationship to Confidence-Based ROCs in Eyewitness Identification. International Psychonomics Society, Granada, Spain, May 2016.
Year(s) Of Engagement Activity 2016
URL http://www.ps2016.org
 
Description International Meeting of the Psychonomic Society 2016 Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact I presented work from the grant at this cognitive psychology conference.

Mickes, L., Wixted, J. T., Dunn, J., Clark, S., & Wells, W. The relationship between confidence and accuracy for eyewitness identifications made from simultaneous and sequential police lineups. International Psychonomics Society, Granada, Spain, May 2016.
Year(s) Of Engagement Activity 2016
URL http://www.ps2016.org
 
Description Invited talk Psychology department University of Cambridge 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact Presented research.

Mickes, L. What eyewitness memory researchers were doing while we weren't looking. University of Cambridge, November 2016.
Year(s) Of Engagement Activity 2016
 
Description Invited talk Psychology department University of Southampton 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact Invited talk about the research.

Mickes, L. Eyewitness Identification: New Findings from Receiver Operating Characteristic (ROC) Analysis, University of Southampton, March 2015.
Year(s) Of Engagement Activity 2015
 
Description Psychonomic Society 55th Annual Convention Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Presented results.

Mickes, L. Eyewitness identification: Which results matter for which decision-makers. Presented at the Psychonomic Society 55th Annual Convention, Chicago, Illinois United States, November 2015.
Year(s) Of Engagement Activity 2015
 
Description Royal Holloway, University of London 2015 Science Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Science festival is open to the public to learn about science and approximately 6000 visitors attend. My lab members, including the postdoctoral fellow, Dr Stacy Wetmore, and I (both funded on the grant) provided activities, demonstrations, gave talks, and conducted experiments related to the grant studies.
Year(s) Of Engagement Activity 2015
 
Description Royal Holloway, University of London 2016 Science Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Science festival is open to the public to learn about science and approximately 6000 visitors attend. My lab members, including the postdoctoral fellow, Dr Stacy Wetmore, and I (both funded on the grant) provided activities, demonstrations, gave talks, and conducted experiments related to the grant studies.
Year(s) Of Engagement Activity 2016
URL https://www.royalholloway.ac.uk/science/sciencefestival/home.aspx
 
Description UCL Crime and Forensic Science Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Seminar speaker
Year(s) Of Engagement Activity 2016
 
Description UCL Psychology Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact Seminar speaker to UCL Department of Psychology.
Year(s) Of Engagement Activity 2016