Detecting nonverbal symptoms of schizophrenia and depression, utilising Computer Vision and Machine Learning methodologies

Mental health is one of the leading causes of disability worldwide with depression being the second largest cause of disability worldwide in 2010 [1]. If left untreated it can lead to serious consequences often suicide [2]. Even though mental health awareness is on the rise we still do see underdiagnosis.
One of the major contributors to underdiagnosis, for example in catatonia, is access to diagnosis and treatment [3]. The most significant factor in mental health underdiagnosis or misdiagnosis is bias in mental health assessment [4, 5]. These show a need for automated and consistent diagnosis for mental health illnesses. Such an approach would facilitate access to diagnosis as well as ensure an unbiased approach to it. As such, the past few years have seen interest from the machine learning research community in mental illness symptom severity estimation, following innovation in other biomedical fields.
Mental illness can manifest with non-verbal symptoms, including body and head movement, eye movement and facial expressions as mentioned by ECSI framework [6]. Patients with depression have been observed to avoid eye contact, smile less and with lower intensity and slower movement [7]. Similarly, patients with schizophrenia tend to avoid eye contact and show more eye closures [6]. Strong nonverbal cues are an additional motivation for the creation of an automatic methodology to detect and assess mental illness as they are harder to measure and record with naked eye; furthermore subtle symptoms (or the lack of symptoms) are harder for therapists to objectively define, introducing bias in diagnosis. Specifically for behavioural symptoms it is estimated that accurately and objectively quantifying them is ten times more time-consuming than filling a rating scale, therefore important information around behaviour is lost or misinterpreted [6].
This work will focus on detecting nonverbal symptoms of schizophrenia and depression, utilising Computer Vision and Machine Learning methodologies. The first stage will focus on predictions from individual modalities, specifically body pose and movement. It will then investigate fusion techniques to connect the different modalities with existing face modality and improve overall prediction accuracy of the model. Finally, it will look into explainable AI techniques to temporally and spatially localise the factors that contribute to the systems prediction.

[1] Alize J Ferrari, Fiona J Charlson, Rosana E Norman, Scott B Patten, GregFreedman, Christopher JL Murray, Theo Vos, and Harvey A Whiteford,"Burden of depressive disorders by country, sex, age, and year: findingsfrom the global burden of disease study 2010,"PLoS medicine, vol. 10, no.11, pp. e1001547, 2013.
[2] European Commission, "Improving the mental health of the population:Towards a strategy on mental health for the European Union," Report,European Communities Bruselas, 2005.
[3] K Adorjan, P Falkai, and O Pogarell, "Catatonia in clinical reality: underdiagnosed and forgotten,"MMW Fortschritte der Medizin, vol. 161, no.Suppl 7, pp. 7, 2019.
[4] Lonnie R Snowden, "Bias in mental health assessment and intervention:Theory and evidence,"American Journal of Public Health, vol. 93, no. 2,pp. 239-243, 2003.
[5] Barbel Knauper and Hans-Ulrich Wittchen, "Diagnosing major depression in the elderly: evidence for response bias in standardized diagnostic interviews?,"Journal of Psychiatric Research, vol. 28, no. 2, pp. 147-164, 1994.
[6] Alfonso Troisi, "Ethological research in clinical psychiatry: the study of nonverbal behavior during interviews,"Neuroscience & Biobehavioral Reviews, vol. 23, no. 7, pp. 905-913, 1999.
[7] Stefan Scherer, Giota Stratou, Marwa Mahmoud, Jill Boberg, Jonathan Gratch, Albert Rizzo, and Louis-Philippe Morency, "Automatic behavior descriptors for psychological disorder analysis," in2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recog-nition (FG). IEEE, 2013, pp. 1-8.