Internet Move Brain (IMBD): Using movies and machine learning competitions to understand how the brain supports natural behaviour

We have little to no scientific understanding of how the brain operates in natural conditions or how various rain networks that support extended real-world behaviours, like language comprehension or emotional processing, organise and interact. Overcoming this issue would inform future much-need developments in Artificial Intelligence and in how we diagnose mental illness, while also contributing to the developments of mathematical approaches in other areas of biology that deal with highly complex systems like brains. The current project proposed that part of the solution of having a sufficient amount of data (of the right type) and developing new mathematical approaches that better consider the complexity of brain function. We can achieve this by generating the first functional magnetic resonance imaging (fMRI) database from people engaged in natural behavioural functions as elicited through full-length movies. the Scope of the project extents to automatically annotating movies, making eh behavioural, fMRI and annotation data publicly available, crowdsourcing analyses via machine learning competitions and making use of the data to explore real-time brain-computer interaction and the design of new digital interfaces. Much progress towards understanding brain function has been made through neuroimaging experiments by decomposing general behaviours into discrete processes that can be associated with activity in particular brain regions. However, a new approach in cognitive neuroscience is needed to produce foundational advances in this field, instead of incremental knowledge. The core of this project is to focus on studying brain function at the macroscopic level by developing appropriate mathematical models to study event-related data from a biological system. The Human Connectome Project, which provides data from 1200 participants scanned for an hour 'at rest' has helped us advance our understanding of functional connectivity via fMRI. The current project would produce similarly valuable data for the cognitive neuroscience community, allowing one to study the organisation of brain networks while engaging in natural processes as elicited by movies. Participants will watch one of 100 uncut English language movies that they have not previously seen. These include ten carefully chosen movies from each of the ten genres (action, comedy, drama, fantasy horror, musical, mystery, romance, sci-fi, war). Movies will have scored highly on a metric of success that considers sales and aggregated reviews. The full database would ideally consist of 600 participants. By the end of a 4-year doctoral project, a medium-sized database is achievable, as well as running yearly machine learning competitions to distribute the process of finding the best methods to decode brain networks associated with annotations of the moves. Behavioural, fMRI and annotation data will be made publicly available through a custom web application. This website will also have the tools to browse and conduct further analyses with decoded fMRI results.These innovations will rapidly accelerate needed advances in understanding how the human brain operates under natural conditions and has medical, education and commercial applications. Aside from data collection, the project involves advancing two exciting areas: automatic annotation of full-length movies and graph theory models of real-life brain function. The completion of the multi-disciplinary project will provide a new way of approaching cognitive neuroscience questions and an appropriate amount of informative data to start understanding the brain in real life conditions. Throughout, mathematical and biological techniques will be of equal use and will provide a proof of concept that we can fundamentally change how we explore brain functioning by considering lessons learnt in neuroimaging in the past 30 years and translating these insights into practical applications to be used outside of the lab.