Learning Cue Combination

People often have to deal with multiple streams of information at once. For example, imagine you are on a walk out in the woods trying to find a kind of rare bird. You hear a bird call (audio information) and turn towards it. You can see some leaves moving around in a tree (visual information). Neither stream of information is perfectly reliable -- you will tend to make some error one way or another when you try to pinpoint the sound location, and you don't know exactly which leaf the bird was behind -- but both are useful pieces of information. What we see in the lab when we give these kinds of tasks to adults is called 'optimal cue combination'. Adults tend to combine all the different 'cues' available in an 'optimal' way that gets them as close as possible to the right location. To do this, they have to take into account how reliable each cue is, weight each cue by its reliability, and then take a weighted average. Developmental Psychologists have found that children don't begin doing this until they are about 10 or 11 years old; before that, they seem to just ignore one cue or the other (e.g. Nardini, Bedford & Mareschal, 2010). This is surprising because in these studies, children have all the information they need to make more accurate judgments. They are just failing to combine the information in the right way. We want to know why. What is changing at 10-11 years old that allows them to start doing optimal cue combination?

We are going to examine two big ideas that might provide good answers to this puzzle. First, children at 10-11 might gain a new ability. They might first develop the ability to learn how to put cues together at this age. The second idea is that it might come down to the quality of the individual cues that children are trying to average together. It's generally a bad idea to do 'optimal cue combination' with a cue that is strongly biased (systematically incorrect) -- in that case, it makes more sense to ignore the biased cue. Even if a cue is not strictly-speaking biased, it might also be so noisy that it is hard to learn how it works. It could be that children under 10-11 years don't show optimal cue combination because the individual cues that they have available are all too biased or noisy. To test these ideas, we will test several predictions that they make. For example, we should be able to train children at 7-9 years as much as we want and they should never learn cue combination; we should be able to prevent adults from learning cue combination with a new cue by inducing child-like biases. It is possible that both of these big ideas might be correct -- it could be that children need a new ability and that the individual cues also have to improve.

This work will be made possible by a combination of methods newly developed in our lab especially for this project, including testing and training children's abilities to combine the senses, and to learn a completely "new" sense, in immersive virtual reality. The work will be interesting to cognitive scientists, especially those interested in development and education. It will help us design better future interventions to teach additional senses to children with sensory loss. It will give us a better understanding of how the use of multiple cues responds to training in childhood, which is of interest to safety-oriented organizations since so many behaviours like safe road crossing can rely on multiple senses (seeing and hearing traffic). It will also be of interest to people who design educational tests since we will continue to develop our new test of a major sensory milestone and show people how similar efficient tests can be designed in other areas of education.

Our impact will have 3 aims: (1) understanding how and when we can improve sensory learning in non-clinical populations in late childhood so that we can strengthen ties and develop strategies for future research into direct applications to clinical populations, particularly vision loss, (2) improve our understanding of the multisensory aspect of basic safety during tasks like road crossing in early childhood and communicate both the current state of the art and our new findings to councils, (3) create a specific worked example of a new individual-level Bayesian psychological test that will be shared via our ties to the educational evaluation and monitoring community at Durham University and online.

(1) Syndromes like retinitis pigmentosa can affect young children by causing gradual permanent loss of vision and even eventual blindness. Echolocation and other supplementary sensory strategies could potentially help affected patients, but current research suggests that a major roadblock exists: children under 10 years old have not been found to use multiple sensory cues simultaneously to improve on the precision they exhibit with just one cue. If this is true then teaching a young child with some remaining vision to echolocate may be less effective than hoped for, as they may fail to supplement their failing vision with echolocation and instead switch between using one sense and the other. This initial training still has the potential to be useful, but to optimise these kinds of interventions we require a better understanding of exactly what is blocking a typical younger person from combining multiple cues. This will allow us to design future applied research with clinical populations to potentially overcome those specific obstacles. We are starting with non-clinical populations in this project but can use results to guide specific research questions in future with smaller, potentially more vulnerable clinical populations.

Objective 1.1: International workshop held at 33-36 months where we communicate findings to the vision loss/sensory substitution community and receive feedback from them about the kind of applied research that they would like to follow; 1.2: online publication of materials.


(2) Many basic safety behaviours require integrating multiple sensory cues in order to be as accurate and safe as possible e.g. crossing the road can require a child to combine cues like peripheral vision and localised sound. Councils should be aware of the danger of assuming that a young person would be able to fall back on secondary cues in an adult-like way and will want to know what kind of training (if any) can improve this.

Objective 2.1: Meeting with Durham and Stockton councils in months 18-21 where we explain the current state of the art and also the results of Studies 1.1-1.3, which all deal with training at different ages; 2.2: online publication of materials.


(3) Educational evaluation and monitoring experts are always facing a crucial tradeoff between the time taken to perform an evaluation and the accuracy of that evaluation. The new analysis introduced in this project shows how Bayesian model-based comparison can allow a specific stage of development to be assessed efficiently within a single child's dataset. The impact here is potentially both specific, in that it assesses a major milestone in sensory development, and general, in that the analysis logic could be used as a template in other assessment settings.

Objective 3.1: On-campus workshop to show educational testers our experimental task, explain the logic of the data analysis (compared with previous approaches) and explore other applications, in months 18-21; 3.2: online publication of materials.