Modelling dynamic cultural variance in social face signals

Humans are a sophisticated social species, frequently engaging in complex dynamic social interactions. A powerful tool for such interactions is the face, which can generate myriad complex dynamic patterns - i.e., facial expressions (Jack & Schyns, 2015) - and thus many nuanced social messages including emotions (e.g., Jack et al., 2016), cognitive states (e.g., Chen et al., 2015), and personality traits (e.g., Gill et al., 2014). Given their centrality in social interactions, a longstanding goal in the field has been to understand the "language" of facial expressions - i.e., which specific syntactical combinations of face movements transmit meaningful social messages to whom - and whether and how this varies across cultures (e.g., Darwin, 1872/1999; Jack, 2013). Yet, to date, little is known about how the face generates and transmits the fundamental and complex social concepts that structure social interactions and uphold broader societal functioning. Such knowledge limitations in turn impede resolution of longstanding debates within the field, such as whether these complex face signals are universally understood or culture-specific and how they map onto psychological processes (e.g., categorical versus dimensional perception). This is largely due to fragmented research on social concepts (mental states, personality, emotions), culture, and the traditional use of Western-centric, theory-driven approaches and static stimuli (e.g., Ekman & Friesen, 1982), which, consequently, has overlooked a possible latent algebraic, syntactical structure to social face signals across cultures. Recent work from Prof. Jack's laboratory hints at such a structure, suggesting that facial expressions follow a latent, coarse-to-fine syntactical structure over time with early-onset, culturally common face movements conveying broad dimensional information and later-onset, culture-specific accents refining the message into more precise social categories (Jack et al., 2014; Jack et al., 2016). Aims. My project draws from such evidence and aims to derive the first generative, algebraic, and syntactical model of social face signals using methods combining social/cultural psychology, 3D dynamic computer graphics, vision science psychophysical methods, and mathematical psychology. Specifically, I will seek to first broaden knowledge of facial expressions to social messages beyond basic emotions both within and across cultures using state-of-the-art, data-driven methodologies developed in Prof. Jack's laboratory. I will then formally examine these facial expression models to extract their temporal structuring, identify specific early- and late-onset face movements, and characterize the dimensional and categorical information they transmit over time using novel information-theoretic analyses (e.g., Ince et al., 2017). Lastly, I will conduct cross-cultural comparisons of these facial expression models to identify the specific face signals that are cross-cultural, those that are culture-specific accents, and test their impact on cross-cultural communication. This project will therefore produce the first ever culturally sensitive syntactical model of facial expressions that will precisely characterize latent facial movement patterns, their culture-specific accents, and the social information (e.g., dimensions, categories) they convey over time, with the potential to bridge longstanding debates in the field (e.g., essentialist versus constructivist; categorical versus dimensional) and form the basis of a new theoretical framework of social face perception that unifies knowledge on social messages, face signalling, and culture.