Understanding and awareness: The roles of conscious awareness in language processing, development and disorders.

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Philosophy Psychology & Language

Abstract

In 1957, the advertising executive James Vicary gathered reporters to announce a startling finding. He had taken movie reels from a local cinema and repeatedly inserted single frames containing simple messages: "Eat popcorn" or "Drink Coca-Cola". The frames were essentially invisible, rushing by too fast for anyone to see, but their effects were extreme: A huge increase in sales at the concession stand. Vicary's finding suggested a powerful role for the unconscious in our everyday lives, and a lucrative new method for advertisers. There was just one problem: He was never able to corroborate the data. His startling result was false.

In the intervening fifty years, we have learned a lot about the unconscious. Techniques for "masking" the world from consciousness have revealed the complex cognitive processes that proceed without awareness. But the role of awareness in language -- our primary means of understanding the world and making ourselves understood -- remains surprisingly unexplored, perhaps a legacy of Vicary's controversy.

This project aims to correct that imbalance, investigating how understandable linguistic meanings arise from the combination of language and awareness. I want to understand the degree to which awareness of the world is a precondition for understanding and producing language. This topic is important in many different ways. It is important for science: Language and consciousness are two critical components of the human experience; understanding their interrelation can help us understand ourselves. It is important for society: Delineating the role of conscious awareness in understanding and being understood can help us to make sense of, and predict, people's behaviour. Finally, it is important for healthy development: As children grow, they have to develop the correct relationship between language and awareness. Understanding how this process might go wrong could improve the lives of both typically and atypically developing individuals.

My approach is broadly focused and experimental in nature. The project examines how language and awareness interact in healthy adults, typically developing children, and individuals with schizophrenia, a developmental disorder often associated with impaired awareness. I use sophisticated experimental techniques to mask sentences from awareness, and then test the degree to which adults can still extract some understanding. I use eye tracking to measure what things in the world children, patients and healthy adults are aware of, and then test whether differences in awareness can explain some of the difficulties that both children and patients have in crafting clear, understandable descriptions of the world.

The results should be important for all of the reasons set out above: They will inform both scientific theories, and methods for alleviating linguistic difficulties. And they will set the stage for future work where, in collaboration with others, I can complement our initial measurements of behaviour with an assessment of the twin neural underpinnings of language and consciousness.

Planned Impact

Who will benefit from the research?
Language and awareness are intimate parts of our daily lives. Our ability to communicate our conscious experiences, and to understand the experiences of others, is core to our understanding of what makes us human. A scientific exploration of the interrelations of these domains should therefore be of interest to a variety of stakeholders. Meanwhile, the research is likely to produce outcomes of tangible importance to educators, clinicians, and anyone interested in influencing choices and behaviours.

1) Influencing choices and behaviours.
In 1957, the advertising executive James Vicary claimed that subliminally presented sentences ("buy Coca-Cola") influenced consumer purchases. That result did not stand up to scrutiny, but our pilot data suggests that unconscious messages might still be processed. Potentially, the fruits of that processing could influence future choices and behaviour, which would be of obvious interest to a variety of groups, from businesses wooing customers to public services. An understanding of this issue also fits the ESRC's strategic priority of understanding how to influence behavior.

2) Impact on educators and clinicians.
The ability to speak clearly and precisely is a key part of everyday life; linguistic skills are a core predictor of life outcomes. Our research can shed light on why some individuals have difficulty communicating clearly (whether in childhood or during mental illness), and so should be of interest to anyone who works with these individuals. Importantly, it also opens up a possible path to improving life outcomes, by training attention to bolster language. This should be of interest to speech-language pathologists, as well as clinicians working with psychiatric patients.

3) Impact on the broader community.
The proposed research touches on consciousness, language, and the process by which children learn to talk. These are all topics that fascinate the general public, inasmuch as they touch on fundamental questions about being human. For instance, a recent article in the Guardian newspaper listed an explanation for consciousness as one of "the 20 big questions for science". Similarly, parents are entranced by the rapid progress -- and occasional errors -- that characterise their growing child's language. This level of interest provides an opportunity to educate the public about not only our own results, but also the methods and importance of social science research more generally.

I will engage these stakeholders through both participation and dissemination. For example, individuals and families who participate in our research will gain experience of state-of-the-art science, discovering how scientists can measure cognitive processes even if they are unconscious, or embedded in the mind of a very young child. I will also act to disseminate research through lectures at schools and local museums, by describing findings on a website for the project, and by crafting press releases in association with both the ESRC and the University of Edinburgh press office. Full details are laid out in the Pathways to Impact section.

Evaluating impact
I will measure and evaluate impact through discussion of the results in the media, and interest in the results from stakeholders (measured by attendance at talks and lectures/visits to the project website). I will also solicit anonymous web-based feedback from attendees at etc, in order to judge the success of events.

Publications

10 25 50
 
Description 1. We have studied how children learn to attend to the world in order to speak informatively. For example, using eye tracking, we assessed why children often produce under-informative utterances (e.g., saying "give me the book" when multiple books are present). We found that children, unlike adults, do not check whether their forthcoming utterances might be ambiguous, by proactively gazing around the world, which suggests that children's difficulty with informative communication may be partially driven by limitations to how they control their attention.

2. We have been studying the role of attention in how adults understand multi-word utterances, like "big spotted boat" or "big pink boat". We have shown that attention is necessary in order to understand phrases where words need to be "bound" together in a specific manner, such as understanding that, in the phrase "big spotted boat" the word "big" might modify "spotted" or might modify "boat", but not both at the same time. By contrast, attention is less important for understanding phrases where the modification structure is less important, like "big pink boat". This has implications for designing instructions for human-computer interaction, particularly for high-pressure situations, as we have shown that it is easier and less attentionally demanding to comprehend instructions in which this type of "binding" is minimized.

3. We have been studying the role of conscious awareness in how adults understand sentences, following up on prior reports that certain types of sentences could be understood unconsciously (e.g., a paper in the prominent journal Proceedings of the National Academy of Sciences, by Sklar et al, 2012). However, our initial pilot work using this method suggested that those experimental result may have been false positives, which we then confirmed in a series of careful experiments, computational statistical simulations, and meta-analysis (statistical aggregation of prior scientific findings). Based on this, we have published an article proposing that conscious awareness plays a key role in understanding language.

4. Building on the expertise developed in the previously-discussed project, we have begun applying meta-analytic and meta-scientific methods to developmental psychology research, and infant cognition research in particular. For example, using meta-analysis, we have provided a new profile of how infants decide when to attend to potentially informative linguistic signals, and shown that infants are more likely to learn generalizable patterns from a stimulus, if they perceive that stimulus to be meaningfully relevant to their everyday activities. Given that this ability is present by 7 months, it suggests that infants have a striking skill for distinguishing between meaningful and non-meaningful stimuli.
Exploitation Route Output 1 could potentially have implications for understanding the language difficulties of, e.g., children with Social Communication Disorder.
Outputs 2 and 3 could potentially have implications for designing systems for Human Computer Interaction, and for designing call systems for communication in high pressure situations.
Output 4 could also potentially have implications for understanding learning difficulties in very young infants.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections,Retail

 
Description This work, on how our ability to understand and use language interacts with processes of attention and awareness, always had a basic science perspective, such that economic and social impact was not a major focus of the award. However, our Pathways to Impact envisioned two routes for dissemination, through parents and families for our developmental psychology, and through influencing work in marketing for our psychophysical studies. Parents and families Over the course of the grant, our research team (the PI, the research assistant, and volunteer students) have discussed our work, and the broader aims of developmental psychology, with an array of parents, families and educators, at a range of venues. For example, we have given workshops and public engagement activities at a number of educational institutions, such as at local preschools and at local libraries, as well as at the Edinburgh Zoo (in 2017 and 2018, including one event as part of the Edinburgh Science Festival), the Edinburgh Camera Obscura museum (a museum focused on human perception and visual illusions). These events have included public talks, but have mainly focused on setting up "booths" that allow us to provide hands on demonstrations of the methods that we use, such as eye tracking and psychophysical techniques, and discussing our results with individual families and children. These events were typically attended by hundreds of families over multiple days; although not all of those families will have interacted with our researchers, the level of engagement with our work was typically high - the rate limiting factor for engagement was the number of researchers with which we could staff these events. Marketing and psychophysics Importantly, the results of our psychophysical studies, on whether adults can understand language without being aware of the words, are somewhat non-traditional, and so we are still planning the most appropriate impact activities. Our key finding from these studies was that the prior literature had over-estimated the degree to which people could process stimuli without being consciously aware of them. That prior literature has been quite influential in marketing, but our findings suggest that many of the key datapoints there are not replicable, such that they are a poor basis for making marketing decisions. We are in the process of collating our results, and those of others, to write a general interest article outlining why these findings should not be used as a basis for important decisions.
First Year Of Impact 2017
Sector Education,Culture, Heritage, Museums and Collections
Impact Types Cultural

 
Description ESRC-NSF SBE RCUK LEAD AGENCY
Amount £538,000 (GBP)
Funding ID ES/N005635/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 07/2016 
End 06/2019
 
Description Newton Advanced Fellowship
Amount £74,000 (GBP)
Organisation Newton Fund 
Sector Public
Country United Kingdom
Start 03/2018 
End 03/2020
 
Description Research Project Grant
Amount £158,500 (GBP)
Funding ID RPG-2014-253 
Organisation The Leverhulme Trust 
Sector Academic/University
Country United Kingdom
Start 11/2014 
End 10/2017
 
Title Understanding and awareness database 
Description This database contains the diverse experimental and meta-analytic datasets collected during the Understanding and Awareness project. The datasets assess psychological research questions involving the relationship between how we understand and use language, and how we attend to the world around us. For example, one dataset investigates whether words and sentences can be partially understood when they have been masked from conscious awareness. Another dataset investigates how preschool children allocate their attention when describing scenes that require them to use potentially ambiguous language. Note that this project did not collect one large dataset, but rather a range of different datasets, with many different characteristics; fuller descriptions of each dataset are provided in the uploaded documentation file. The datasets in this deposit report 1) Chronometric (response time) studies conducted with adults and with preschool children (aged 3 and 5). 2) Eye tracking studies conducted with adults and with preschool children (aged 2 through 5). 3) Psychophysical (continuous flash suppression) studies conducted with adults. 4) Looking time experiments conducted with infants (age 7 months). 5) A database containing records of a meta-analysis of infant looking time data. The size of the datasets range from a meta-analysis containing approximately 100 records, to a collection of psychophysical datasets containing thousands of records for 100s of subjects. The DOI's for the published datasets can be found at 10.1111/desc.12704; 10.1037/xge0000348. 10.1016/j.cognition.2017.01.013 10.1016/j.jml.2016.09.007 10.1016/j.cognition.2016.10.003 10.1177/0956797618822802 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
Impact These datasets have been used in a series of recent publications. Notable impacts outside academia are to come. 
URL http://reshare.ukdataservice.ac.uk/853404/
 
Description Collaboration with Mahesh Srinivasan on lexical flexibility 
Organisation University of California, Berkeley
Department Department of Psychology
Country United States 
Sector Academic/University 
PI Contribution With Mahesh Srinivasan at UC Berkeley, we have designed a series of studies to assess the development of children's understanding of ambiguous words. This is a fully collaborative partnership, where both groups contributed equally to designing experiments and collecting data, and writing the grant that was subsequently funded by the ESRC/NSF.
Collaborator Contribution This is a fully collaborative partnership, where both groups contributed equally to designing experiments and collecting data.
Impact Rabagliati, H. & Srinivasan (in press). Sense Relations. In Cummins, C. & Katsos, N., (Eds.), The Oxford Handbook of Experimental Semantics and Pragmatics. Oxford, UK: OUP. Srinivasan, M., Berner, C. & Rabagliati, H. (2016). Children's use of polysemy to structure new noun categories. Talk presented at the 41st Boston University Conference on Language Development, Boston, MA, November 4-6, 2016. Rabagliati, H., Conte, S. & Srinivasan, M. (2015). Words as invitations to form categories? The case of polysemy. Poster presented at the 40th Boston University Conference on Language Development, Boston, MA, November 13-15, 2015.
Start Year 2014
 
Description Public engagement talks 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact We have participated in a number of public engagement sessions at local libraries and events for young children, such as with the Edinburgh Book Bugs groups.
Year(s) Of Engagement Activity 2016,2017,2018
 
Description School visits 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact We have given a number of talks about child development to parents and staff at local pre-schools.
Year(s) Of Engagement Activity 2015,2016,2017,2018,2019
 
Description Wee Science at the Camera Obscura 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact We set up a workshop at the Camera Obscura museum in Edinburgh during their Teeny Tiny Toddler Fest, a week-long event for parents of young children. We discussed our work and the science of child development.
Year(s) Of Engagement Activity 2015,2016,2017,2018,2019
 
Description Wee Science at the Edinburgh Zoo 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact We have set up a regularly attended "Research Station" at the Edinburgh Zoo, where we talk about our work to interested families, and give children the chance to participate in our studies of child language development and cognition.
Year(s) Of Engagement Activity 2016,2017,2018