ORA (Round 5): The active observer

Every time we move, the image of the world at the back of the eye changes. Despite this, our perception is of an unchanging world. How does the brain translate a continually changing image into a percept of a stable, stationary, rigid world? Does the brain use a map of the external environment (an "allocentric map") and the position of the observer within it, built up over time, to underpin the perception of stability? Does the brain continually update a map of where scene objects are relative to the observer (an "egocentric map"; e.g. there is an object straight ahead of me, if I walk forward I should expect it to get closer to me)? Does the brain not create a map but just divide up the image motion into that which is likely due to movement of the observer (and which can consequently be ignored) and that which is likely due to objects moving within the scene (which become a focus of attention)? The hypothesis that underpins this research project is that it is not a single one of these mechanisms that underpins perceptual stability, but that all of them, their contribution dependent on the task being performed by the observer. In some cases the task will require a fast estimate to support an ongoing action which might favour one mechanism, on another task, where timing is not so critical, a slower, but more accurate, mechanism might be more appropriate. This collaborative project, which combines complementary expertise in Psychology, Movement Sciences, and Computing from Germany, The Netherlands and the United Kingdom, and importantly, researchers that start from different theoretical perspectives, will test this hypothesis. We will study a diverse series of tasks that present a range of challenges to the moving observer. We will make use of various innovative experimental paradigms that exploit recent technological advances such as virtual reality combined with simultaneous motion tracking. Understanding where and how different mechanisms of perceptual stability play a role advances not only our scientific understanding, but also has the potential to inform industry as well as medicine about the circumstances in which disorientation or nausea in real or virtual environments can be minimised.

Industrial impact
Perceptual stability is a key problem in the development of artificial virtual environments, as is illustrated in this recent account of the problems that people have moving through virtual environments (https://blog.google/products/google-vr/daydream-labs-locomotion-vr/). Members of the team have established contacts with industrial groups (e.g. Oculus, Sony) working on these problems and have a long track record of working in or with industry (Hewlett-Packard, Nissan). A better understanding of the different possible visual representations and when we use them is als likely to make systems visually more intuitive, and therefore easier to use (as has been demonstrated for haptics: Mugge et al., 2016).
A 4 day "impact" residential workshop will be held in the UK in the second year of the project. This is modelled on a similar workshop run last year with Microsoft Hololens and HP labs participants.

Clinical impact
An understanding of the underlying mechanisms of perceptual stability has important implications for several clinical conditions of visual vertigo and nystagmus. For example, we will share our findings with Prof. Jelte Bos, an expert on motion sickness who advises on motion sickness in many contexts, including space travel, seasickness, entertainment (3D movies) and self-driving cars. The UK PI is also collaborating with clinical and academic colleagues on understanding visual vertigo, a clinical condition in which visual motion is perceived as overwhelming in some situations (http://psych.cf.ac.uk/engagementimpact/visualvertigo/ , http://www.bbc.co.uk/news/uk-wales-south-east-wales-38715719). Any findings with clinical relevance will be presented at a suitable UK based meeting.

Popular interest
The topics covered in this proposal will likely raise public interest. Perception and action in real and virtual environments attracts attention of a broad audience and can be comprehensible even for laypeople. Likewise these topics are well received in popular science magazines, national newspapers and television programmes. The 3 PIs have contributed to such public outreach in the past, e.g. Gehirn und Geist (German PI), BBC Wales Today and BBC Points West (UK PI). Moreover, yearly science fairs organized by the universities and local museums (e.g. Cardiff University's Brain Games, http://sites.cardiff.ac.uk/cubric/public-outreach/brain-games which attracted almost 4000 people last year) provide an excellent opportunity to reach a broad audience. We will make use of such opportunities to communicate our scientific findings to the non-academic community.

Training Highly Skilled Researchers
We will train post-doctoral fellows who will have substantial input into the organization and design of the project. These postdoctoral fellows should gain the skills necessary to run their own laboratories and translate the techniques and findings into applied settings.