Characterising Motor Impairments In Autism Using Imitation And Computational Techniques
Lead Research Organisation:
University of Manchester
Department Name: Electrical and Electronic Engineering
Abstract
The main objective of the project is to create machine learning algorithms to help identify autism using motion trackers. Potential implications of such work could be far reaching. Diagnosing autism is a long and complex process and a third of families wait more than three years for a diagnosis. Possibility to identify autism using movement characteristics instead of currently used language and social behaviour characteristics could speed up this process. Moreover, diagnoses potentially could be made earlier in the child's development as motor functions emerge before language and most social behaviours. Investigating movement differences is promising. Evidence shows that various motor impairments and especially praxis deficits are common, occurring in >70% of the people with autism. Although not all evidence is consistent, generally large effect sizes are found in controlled laboratory based tests. Recently, there is a great research interest in motor imitation in autism. Motor imitation deficits appear to be more consistent and are likely caused by top-down modulating processes not so much by the impaired ability to perform movements. Ideally, the task which will be used in the project for discriminating between autistic and healthy individuals will capture both, the ability to perform a movement, and the ability to imitate. Developing such task will be one of the main challenges of this project. Numerous theories which aim to explain movement and imitation differences exist but none of them appear to be able to explain all of the available evidence. It is likely that motor and imitative differences in autism are caused by a complex interaction of multiple factors throughout the development and these differences are heterogeneous. Machine learning methods are well suited for detecting interactions between multiple factors and even possibly for exploring heterogeneity. Indeed, another main challenge of this project will be creating new statistical, feature selection and machine learning methods to identify the most discriminative movement features that optimally describe the characteristics of different groups
Organisations
Publications
Gowen E
(2020)
Instructions to attend to an observed action increase imitation in autistic adults.
in Autism : the international journal of research and practice
Vabalas A
(2019)
Machine learning algorithm validation with a limited sample size.
in PloS one
Vabalas A
(2019)
Kinematic features of a simple and short movement task to predict autism diagnosis.
in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509565/1 | 01/10/2016 | 30/09/2021 | |||
| 1830965 | Studentship | EP/N509565/1 | 01/10/2016 | 31/03/2020 | Andrius Vabalas |
| Description | Behavioural results Coordination difficulties are widespread in autism, both in terms of affecting a range of movements as well as affecting the majority of autistic individuals. Accumulating evidence shows reduced accuracy of imitation in autistic children and adults: although autistic individuals are able to imitate the goal of an observed action they are less accurate at imitating its style (e.g. speed, size). Here, we investigated whether (1) reduced visual attention to the observed action might account for altered imitation in autistic adults (2) imitation and coordination ability could differentiate autistic from non-autistic people. 22 autistic and 22 non-autistic participants observed, then imitated sequences of hand movements while their eye and hand movements were tracked. All participants performed imitation after a general instruction to "copy the action," and after explicit instructions to attend to the movement features (e.g. height). Group differences in imitation patterns and a simple aiming task were explored using Machine Learning. Explicit instructions improved imitation of action style in the autistic group to levels that were similar to the non-autistic group. Additionally, the autistic group spent significantly less time looking at the hand movement across conditions. These findings suggest that altered attention to the observed action contributes to differences in imitation in autism with implications for how autistic people learn and understand the actions of others through observation. The content of this study has been published Machine learning algorithm validation with a small sample size To be able to implement robust machine learning analyses in \textbf{Chapter~\ref{Second}} we assessed commonly used ML result validation methods. Our literature survey of ML classification studies in autism research showed a strong negative relationship between sample size and reported classification accuracy. Theoretically the opposite should be the case, because generally larger sample sizes bring higher power to detect regularities in the data. We have investigated weather this discrepancy could be due to validation methods which insufficiently control noise fitting in the data. The results showed that commonly used result validation methods which do not fully separate training and validation data produce biased (inflated) performance estimates and the bias is higher with small sample sizes. The methods which avoid training and validation data pooling produce unbiased performance estimates regardless of sample size. We demonstrate that even an economical method which reuses all of the training data for validation in nested cross-validation fashion produces unbiased results. The content of this study has been published Machine learning results on the imitation data We used kinematic and eye data from the imitation experiment to classify autistic and non-autistic individuals by employing SVM coupled with feature selection. The aim of the study was a reliable ML application. To avoid overfitting, we split 1/3rd of the data as a holdout set and developed models using Nested cross-validation. We have also developed feature selection methods aimed at selection stability to assure result interpretability. The results showed that combining kinematic and eye data provided complementary information and gave the highest classification performance compared to using kinematic or eye datasets alone. Consistent with the behavioural results, the most discriminative features were from the experimental condition in which non-autistic individuals tended to successfully imitate unusual movement kinematics while autistic individuals tended to fail. The content of this chapter is under review. Machine learning results on motor function data we have used kinematic data from the same simple and quick to perform motor function task and applied different classification approaches. In the first study, an SVM was coupled with feature ``engineering'' and selection. In the second study, for classification, instead of using several time discrete features, we used all data and DNN's, which automatically infer high-level data representations for classification. There were notable similarities between studies, classification performance was very similar, and for both approaches, group mean differences played an important role in classifier decisions. With DNN we have used a recently proposed Layer-wise Relevance Propagation (LRP) method to explain what in the data was important for classifier decisions. Instead of exploring why individual samples were classified in a particular way, we statistically investigated data relevance contributions from all samples. We found that the beginning and the end parts of the movement were most relevant for classification and this fitted well with the evidence from movement research in autism. We were also able to demonstrate that the mean differences between groups in the input data were important for the classifier to correctly separate groups. The content of the first study has been published and the content of the second study is in preparation for publication |
| Exploitation Route | In this project, we have demonstrated that even with small sample data, which is unusual and difficult for ML application, statistically significant discrimination between classes is achievable. To take this research forward it would be necessary to collect larger data samples to assure greater pattern recognition capability and result reliability. This can be achieved by combining cheap data collection methods (such as internet of things (IoT) devices) with developed theory-driven tasks which both would provide discriminable data and large sample sizes, enabling the development of ML algorithms with a real potential of diagnostic applications. Thinking about further future - with data collection becoming more widespread, automated and cheaper, advancements in ML methods, and with hypothesis-driven research identifying more precise biomarkers, there is little doubt that in the future technology-based automated methods will aid experts in screening and diagnosing disorders. |
| Sectors | Electronics,Healthcare |
| Title | Kinematic and looking behaviour features of a movement imitation task |
| Description | A dataset generated during a study: A. Vabalas, E. Gowen, E. Poliakoff, and A. J. Casson, "Applying machine learning to kinematic and eye movement features of a movement imitation task to predict autism diagnosis, "Scientific Reports, (in press). The data set contains features calculated using motion tracking and eye-tracking data used to classify 22 autistic and 22 non-autistic individuals. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2020 |
| Provided To Others? | Yes |
| Impact | A study: A. Vabalas, E. Gowen, E. Poliakoff, and A. J. Casson, "Applying machine learning to kinematic and eye movement features of a movement imitation task to predict autism diagnosis, "Scientific Reports, (in press). |
| URL | http://dx.doi.org/10.17632/fnt6jtc5np.4 |
| Title | Python code used for simulations to assess different machine learning validation methods |
| Description | The simulations showed that commonly used K-fold cross-validation method is not sufficient to control noise fitting in the data and produces biased performance estimates, with a stronger bias when the sample size is small. In contrast, Nested cross-validation ant train/test split approaches produce unbiased performance estimates regardless of the sample size |
| Type Of Material | Computer model/algorithm |
| Year Produced | 2019 |
| Provided To Others? | Yes |
| Impact | ML work which resulted in two publication was carried out using robust result validation methods: A. Vabalas, E. Gowen, E. Poliakoff, and A. J. Casson, "Applying machine learning to kinematic and eye movement features of a movement imitation task to predict autism diagnosis, "Scientific Reports, (in press). A. Vabalas, E. Gowen, E. Poliakoff, and A. J. Casson, "Kinematic features of a simple and short movement task to predict autism diagnosis," in2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2019, pp. 1421-1424. DOI:10.1109/EMBC.2019.8857307.10 |
| URL | https://doi.org/10.1371/journal.pone.0224365.s001 |
| Description | Facilitator in a "Postcards from an Aspie World" event |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Patients, carers and/or patient groups |
| Results and Impact | Acted as a facilitator in a "Postcards from an Aspie World" event, The Manchester Museum (March, 2018) |
| Year(s) Of Engagement Activity | 2018 |
| Description | Oral presentation at RIO 2018 group meeting |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Oral presentation at RIO 2018 group meeting: "Characterizing Motor Impairments In Autism Using Imitation And Computational Techniques". Presented imitation and eye movement results from my first experiment. Bielefeld, Germany (April, 2018) |
| Year(s) Of Engagement Activity | 2018 |
| Description | Oral presentation for A-level Singaporean students visiting The University of Manchester |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Schools |
| Results and Impact | Gave a presentation titled: "Support Vector Machine Method and Its Application for Single Subject Prediction of a Clinical Disorder" for A-level Singaporean students visiting The University of Manchester, April 2017. (1h) |
| Year(s) Of Engagement Activity | 2017 |
| Description | Poster presentation PGR Research Conference |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | Poster presentation "Characterising Motor Impairments in Autism Using Imitation and Computational Techniques" in the PGR Research Conference, School of Electrical and Electronic Engineering ,The University of Manchester, November 2017. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Poster presentationin the Postgraduate Summer Research Showcase |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | Poster presentation "Identifying Autism From Movements: Using Motion Tracking and Machine Learning Methods" in the Postgraduate Summer Research Showcase, The University of Manchester, June 2017. |
| Year(s) Of Engagement Activity | 2017 |
| Description | Research van with video and poster presentations about autism research |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | Helped to run a research van with video and poster presentations, short research tasks and questionnaires and printed material (newsletters, information about on-going experiments) for raising awareness about autism research in Autism Conference 2017: Awareness and understanding. The University of Salford, April 2017. (8h) |
| Year(s) Of Engagement Activity | 2017 |
| Description | Science Uncovered event at The Natural History Museum, |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | Ran activities and talked about my research at Science Uncovered event at The Natural History Museum, The University of Manchester, September 2017. (6h) |
| Year(s) Of Engagement Activity | 2017 |
| Description | Workshop presentation about Support Vector Machine method |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | Workshop presentation "Support Vector Machine Method and Its Application for Single Subject Prediction of a Clinical Disorder" in the inaugural machine learning workshop series in the School of Electrical and Electronic Engineering, The University of Manchester, February 2017. |
| Year(s) Of Engagement Activity | 2017 |
| Description | presenting and helping with organisation in autism@manchester 'experts by experience' group |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Patients, carers and/or patient groups |
| Results and Impact | Gave a presentation at "experts by experience" group 20/09/2017 meeting (autism@manchester) titled: "Investigating Movement Differences in Autism", The University of Manchester, September 2017. (2h) Helping Emma Gowen a chair of autism@manchester to organise three initial 'experts by experience' group meetings happening every three months. The group has an advisory role for researchers to consider the perspectives of people on the autism spectrum and their families when planning research, The University of Manchester, June 2017. (7.5h) |
| Year(s) Of Engagement Activity | 2017,2018 |
| URL | http://www.autism.manchester.ac.uk/connect/expert-by-experience/ |