PDP-squared: Meaningful PDP language models using parallel distributed processors.

Lead Research Organisation: University of Manchester
Department Name: Psychological Sciences


Parallel Distributed Processing (PDP) is a form of computation where a large number of processing units performing simple calculations can be employed all together to solve much more complex problems. Perhaps the best example of this is the human brain, which contains approximately one hundred billion neurones. Individually these neurones simply have to decide whether to fire or not, and they do this based upon how many other neurones that are connected to them have fired recently. When this simple local computation is distributed over billions of neurones it is capable of supporting all the extremely complex behaviours that humans exhibit / talking, reading, walking, running etc / behaviours that are well beyond the abilities of more traditional computers. For this and other reasons, many psychologists believe that PDP models are the best way of describing human cognition. Unfortunately, at the moment these models are invariably simulated using standard PCs, which means that each unit in the model has to be dealt with one after the other in a serial process. This serial processing imposes severe limitations upon the complexity of problems that can be tackled. Our goal is to us to understand how the brain supports language function, how this breaks down after brain damage and the mechanisms that support recovery/rehabilitation. This will require a model of language that is capable of simulating speech, repetition, comprehension, naming and reading. To train such a model using existing pc-based simulators would take far too long /possibly more than a lifetime. So the first objective of this project is to produce a parallel distributed processing machine that is truly parallel (PDP-squared). We intend to use an array of 10,000 ARM processors incorporated into a machine that will be able to run our simulations of human behaviour 500-1000 times faster than is currently possible on a single pc. Once we have successfully produced this machine (Phase1 of the project), we will use it to build a model of normal human language function that can support reading (both aloud and for meaning), comprehension, speech, naming and repetition for all of the single monosyllabic words in English. We will validate this model by showing that damaging it can lead to the same patterns of behaviour as found in brain damaged individuals (Phase 2). Finally we will use the model to predict the results of different speech therapy strategies and will test these predictions in a population of stroke patients who have linguistic problems.


10 25 50
Description We have developed an algorithm that allows for continuous backpropogation MLP networks to be run on the SpiNNAker hardware consisting of a scalable number of chips. We have developed a computational model of reading that links visual, orthographic, phonological and semantic representations in an interactive processing model. Using this model we have made a significant contribution to the literature on reading models by showing how a purely parallel model can account for behavioural effects that were previously thought to require serial processing. Finally, we have explored predictions from the model that multimodal therapy may be more efficient then unimodal therapy at restoring the original representational structure following brain damage. These predictions have been tested in a patient population.
Exploitation Route The clinical findings will be of use to speech therapists and other health professionals when designing individualized patient therapy.
The ability to run large scale MLP models on scalable hardware is potentially useful to a wide variety of applications including data mining, neuroscience imaging and financial modelling. However, further work is required to package the rather complex software more appropriately, before this would be a realistic option.
Sectors Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Healthcare

Description Project Grant
Amount £361,080 (GBP)
Funding ID ES/L006936/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 02/2015 
End 01/2018