An affective system for early diagnosis of mental health disorders

Assistive technology, rehabilitation and musculoskeletal biomechanics - Healthcare technologies

Affective computing (AC) is a field of research focussed on the design and creation of systems or devices capable of recognising, processing, and interacting with human affect. The proposed PhD research will investigate the use of AC technologies for the diagnosis and monitoring of mental health disorders, in particular depression. The research will investigate the development of automated emotion recognition method/s from facial expression and speech, and a sentiment analysis method for the development of a system to help clinicians detect early sign of depression and monitor patients' affective state throughout the rehabilitative treatment. The research will initially use existing publicly available databases, e.g., SEMAINE dataset, and adapting existing and developing machine learning methods to build predictive models of a subject's affective state. The later stages of the system development will involve a physician in the validation process of the system performance. The aim of the PhD research is to design and develop an AC system which identifies symptoms of depression allowing for early corrective action. The research will investigate the appropriateness of affect detection methods, sentiment analysis and behavioural monitoring before selecting/adapting/developing the optimal tools for the system. The modalities to use are facial expressions and speech (spoken and written). The objectives of the PhD research: To investigate multimodal affect recognition approach involving facial expressions and speech to identify early symptoms of mental health disorders. To investigative the use of sentiment analysis of spoken or written words to infer the subject's mental state. To investigate a real-time and accurate method for classifying facial expressions. To investigative the incorporation of the outcome of the above three objectives in the development of an AC system for depression monitoring. Novelties of the research methodology: The combined use of action units of the Facial Action Coding System (for classifying facial expression), acoustic features (for detecting speech parameters that indicate depression or other mental disorders), and multiscale feature fusion to detect depression and their severity. The use of sentiment analysis and machine learning to generate a model for identifying different sentiment expressed in a section of text or speech. The use of manifold learning to recognise depression from facial expressions. The optimising procedure for feature fusion.