Dynamic Mental Representations of Motion Events in Sentence Processing
Lead Research Organisation:
University of Dundee
Department Name: Psychology
Abstract
Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
Organisations
Publications
Dunn, B
(2014)
Conference Paper / Proceedings
in CogSci 2014 Proceedings
Kamide Y
(2016)
Event processing in the visual world: Projected motion paths during spoken sentence comprehension.
in Journal of experimental psychology. Learning, memory, and cognition
Lindsay S
(2013)
To Dash or to Dawdle: Verb-Associated Speed of Motion Influences Eye Movements during Spoken Sentence Comprehension.
in PloS one
| Description | Overview: The present project (Oct 2011 - Nov 2014) aimed to investigate how spatial information is represented in understanding of motion events in language processing. More specifically, our research explored two aspects of motion events, (a) the shape of the trajectory of the movement and (b) the speed of the movement. We conducted 16 main and 3 norming experiments, using a 'visual-world' eye-tracking technique (10 expts), in addition to traditional 'eye-tracking during reading' (1 expt), cross-modal priming (1 expt), eye fixation drift (1 expt), and mouse-dragging (3 expts) methods. [note this report is not concerned with our project-linked PhD studentship]. Findings: Our research offers rather novel approaches to online motion event processing. Our findings can be summarised as follows: (a1) Our research broadly suggests that language comprehenders could form and update representations for the trajectory of a motion event as linguistic input unfolds. For example, participants heard auditory sentences such as "Foodtock will jump / crawl to the sofa", simultaneously looking at visual scenes containing related objects (a made-up creature (Foodtock - agent) and sofa (goal), both in a room). We found higher mean Y-coordinates for 'jump' than 'crawl' in the 'path' region between the agent and goal objects between the verb offset and sentence offset. Our results therefore suggest that the verb trajectory could modulate the locus of comprehenders' attention shifts. (a2) Thus, such operations appear to be consistent with accounts that language comprehension involves a 'mental simulation' of the event or state described in language. (a3) However, when the visual spatial frame of reference was removed, evidence for trajectory effects was not obtained. For example, in a simple text-reading experiment, we did not find evidence that readers' fixations were on average higher when they were reading narratives suggesting an upper movement than a lower one. Similar null effects were obtained in a cross-modal priming experiment (auditory trajectory discourse + visual probe word). (a4) Hence, so far, it seems safe to argue that our results suggest trajectory representations could be established and updated as soon as such trajectory information becomes available when comprehenders have enough contexts (e.g., accompanying visual scenes). When there is no spatial frame of reference, such operations do not seem to occur, or at least not online or not with manipulations we opted for. (b1): Our findings on speed representations similarly indicate dynamic updating of spatial representations in language processing: e.g., language comprehenders eye movements on the path and goal regions were distinguishable when hearing a sentence showing a fast or slow motion event (e.g., "The boy will dash (fast) / stagger (slow) along the road to the house") while looking at relevant scenes (e.g., with a boy, house and passage). (b2): We also partially replicated speed effects even when the speed was implied in certain other ways (e.g., with preceding contexts), but not some other ways (e.g., with visual features of the agent), which opens up deeper issues surrounding the integration of visual and linguistic information in motion event processing. |
| Exploitation Route | Academic impact: We have already published findings from one eye-tracking and one mouse-dragging experiments in a prestigious international journal (Lindsay, Sheepers, & Kamide, 2013; PLoS One). We are also in the processing of revising a draft containing two eye-tracking and mouse-dragging experiment. We might be able to write up another paper on other data, depending on the results from the final experiment. The majority of significant data have been presented at major international conferences (AMLaP, CogSci) during the lifetime of the project. Public engagements: We have been actively involved in presenting our research to a local community, especially to school pupil or college students, at a few public engagement events at the University of Dundee. In these events, the focus was on the basis of the methodologies we use in our research. Wider public impact: This is an area which we have not been engaged in. One possibility is to seek possibilities to give talks to public sectors (e.g., councils, local education authorities), which could lead to useful dialogues. Given the non-applied nature of our research, this pathway would be the most difficult to work on. |
| Sectors | Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Healthcare |
| Description | So far it seems that our research outcome has not been 'used' by other people (educational policy makers etc) in applied settings. Since our research outcome is predominantly academic and generally belongs to a category of basic science, we predict it will take much longer for it to produce tangible impacts. However, we have actively involved in show-casing our research and outcome to school pupils and their teachers on several occasions in the hope that our research will inspire future scientists and current educators (e.g., 'Engage Dundee' in 2013 with an audience of about 150 primary school pupils; several visits to our labs from local high school students in 2014 & 15). We are also in the process of applying our research to different populations, such as normal aging adults, aging adults with dementia, people with no or little language, people with little mobility etc. The difficulty has been that it is not straightforward to make our research outcome applicable to make the society a better place, due to the nature of our research. However, we have a strong belief that efforts to publicise our research and actively communicate with relevant people and organisations will turn out to be worthwhile. |
| First Year Of Impact | 2013 |
| Sector | Education |
| Impact Types | Cultural,Societal,Policy & public services |
| Description | A conference talk (invited) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | "Language-space interface: Dynamic spatial representations of motion events in language processing" - an invited oral presentation at a conference "Event Representations in Brain, Language & Development (EvRep)", Max Planck Institute (MPI) for Psycholinguistics, Nijmegen, The Netherlands, October 2017. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.mpi.nl/events/evrep |
| Description | Engage Dundee 2013 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Schools |
| Results and Impact | Presentation to school children Good feedback from participants |
| Year(s) Of Engagement Activity | 2013 |
| URL | https://www.flickr.com/photos/cassdundee/sets/72157633500897065/# |
| Description | Invited talk at a workshop organised by Scottish Book Trust |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Kamide, Y. (2016). Interface between language and vision: Evidence from eye movements. Paper to be presented at Knowledge Exchange Conference on Neuroscience and Narratives, Scottish Book Trust, University of Strathclyde, Glasgow, UK. The event was organised by Dr Elspeth Jajdelska, University of Strathcylde, and Scottish Book Trust. The main audience was practitioners, such as librarians, school teachers, as well as general public and academics (including PG students). |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.scottishbooktrust.com/blog/reading/2016/05/the-science-of-stories |
| Description | Local and national media coverages |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Although it was not directly related to the current project, an MSc student's project that used the same technique was covered by local media (Courier & Evening Standards) and national media (Daily Mail, Sun). |
| Year(s) Of Engagement Activity | 2014,2017 |
| URL | http://www.dailymail.co.uk/sciencetech/article-4607036/How-English-people-respond-Scottish-slang.htm... |
| Description | School Visit 2014 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Schools |
| Results and Impact | Presentation and demo to secondary school pupils Good feedback from participants |
| Year(s) Of Engagement Activity | 2011,2014 |
| URL | http://blog.dundee.ac.uk/publicengagement/?post_type=casestudy&p=474 |
| Description | School Visit 2014 (no.2) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Schools |
| Results and Impact | Presentation and demo to secondary school pupils |
| Year(s) Of Engagement Activity | 2014 |