Machine analysis of human expressive behaviour
A critical issue in machine analysis of human naturalistic behaviour is that the human face, body and vocalisations exhibit complex and rich dynamic behaviour that is non-linear, time varying and dependent on context. Backed by EU funding, the project MAHNOB (Multimodal analysis of human nonverbal behaviour in real-world settings) set out to address the related challenges and build automated tools for machine understanding of human interactive behaviour in naturalistic contexts. The team's researchers have a background in signal processing and machine learning and were at the forefront of research on body and multimodal naturalistic behaviour analysis. They proposed a number of firsts in relation to approaches to distinguish between various spontaneous behaviours. One example is automatic discrimination between agreement and disagreement episodes on the basis of non-verbal behavioural cues. The researchers also proposed novel and truly unconventional computer vision and machine learning methodologies, including image gradient orientation-based subspace learning. In addition, they extended the hidden conditional ordinal random fields to allow simultaneous recognition of facial expression and their intensities and to account for differences in subjective facial displays. Project work advanced the state of the art in automatic facial behaviour analysis in several directions. These include the accuracy and robustness of face and facial feature detection and tracking, the efficiency and accuracy of automatic recognition of facial muscle actions, and the extent and accuracy of automatic recognition of temporal phases and intensity of facial muscle actions. Four databases, believed to be the first of their kind, of multimodal recordings of human spontaneous behaviour were released. These were captured while subjects were involved in dyadic interactions or watching multimedia material. MAHNOB has made advances in audiovisual spatiotemporal methods for automatic analysis of human spontaneous patterns of behavioural cues. The work has important implications as machine analysis of behavioural dynamics is crucial for analysing and correctly interpreting complex behaviours, including emotions, pain and depression.
Keywords
Expressive behaviour, human behaviour, MAHNOB, nonverbal behaviour, machine learning, behaviour analysis
