Affective Computing Models: from Facial Expression to Mind-Reading

Humans exhibit and communicate with a wide range of affective and cognitive states. Mind reading allows humans to predict, model, and interpret each other's behaviour beyond the capabilities of other animals, a claim that arguably can be made despite recent research suggesting apes being successful with false-belief tasks. Therefore, mind reading is fundamental to human social interaction and communication. In mind reading, one of the most important signs is facial expression, as it conveys critical information that reflects mental states and relates to 55% of information when people perceive others' feelings and attitudes. Since Duchenne5 studied the electro-stimulation of individual facial muscles in 1862 and ten years later, Darwin published "The Expression of the Emotions in Man and Animals", making a case for shared ancestry of facial expressions. Research on facial expressions has attracted a lot of attention from different disciplines such as psychology, neuroscience and computer science. In recent years, the development of computing technologies and massive online facial images/videos enabled the boosting of deep learning-based facial expression recognition (FER). To date, automatic FER has achieved excellent progress, from static image to dynamic video analysis, from acted/posed to spontaneous expressions, from macro-expressions to micro-expressions.

In summary, the rising challenges include,
1) Substantial psychological works support the use of appraisal theories of emotion for internal emotion detection through facial behaviours. At the same time, research in computer science mainly focuses on appearance or geometric facial modelling but ignores the underlying biologically-driven mechanism;
2) There is limited available data from different cultures, hindering the research on machine learning method development;
3) Micro-expressions, rapid (1/25 to 1/3 second), subtle, and involuntary facial expressions that are difficult to control through one's willpower, is not studied for culture inconsistency;