Improving deep neural networks as a modelling framework for human ventral stream computations

The overall aim of my doctoral research is to develop better models of the ventral stream. The standard test of mechanistic similarity between such a model and the brain is the model's ability to predict neuroimaging data from healthybrains. Currently, DNNs can explain more than half the variance of primate electrophysiological data. However, a model of the ventral stream should not only predict healthy neural activations, but should also reproduce human-like errors in response to progressive lesions of the system. Filling in this gap, I am developing a test case that would act as an additional assessment of how brain-like a DNN's visual information processing is. To this aim, I am using semantic dementia (SD) patient data. SD patients have localised lesions in the temporal lobe and exhibit well-defined impairments, for example, subordinate level categorisation is impaired while basic level categorisation is preserved. DNNs that act as models of the human temporal lobe should therefore replicate these impairments when parts of the model are lesioned. Once successful models are found, lesioned DNNs mayalso be useful predictors of the types of judgements and distinctions that may be difficult for SD patients.

The aim of the second stage of my proposed research will be to develop DNNs that better predict both healthy neuroimaging data and patient data. Whilst DNNs are biologically-inspired, they have not been developed with the intention of explaining neural computations, but instead are used most commonly as engineering solutions to computer vision problems. Therefore, it is reasonable to suggest that their architecture, functional objective(s), input training data and learning algorithms could all be improved based on neuroanatomy and neuropsychology in order to achieve biological plausibility. I hypothesise that the inclusion of relevant biological detail will result in more brain-like models. I will initially explore DNNs with a different functional objective; trained to predict an item's position in a high-dimensional semantic embedding space rather than the commonly used categorisation objective. It is unlikely that the visual system's primary goal is categorisation. A category-trained DNN aims to separate the clusters of 'cat,' 'dog' and 'umbrella' orthogonally as far apart as possible in order to be able to differentiate between them, but to be more realistic, the 'cat' and 'dog' clusters should be much closer together than 'umbrella' in the model's feature space. I will therefore train a DNN to predict high-dimensional word embedding vectors, obtained from linguistics in which items with similar WEVs appear in similar written contexts. I hypothesis that these DNNs will be able to better predict both behavioural data from healthy humans and also SD patients when the models are lesioned as part of my new testbed.