Brains and Muscles: Learning about Facial Expressions
and Gestures using EMG and EEG Measurements
Special
Session at the 2008 IEEE International Conference on Automatic Face and Gesture
Recognition (FG 2008),
September
17-19, 2008,
Organizers:
Anton
Nijholt (
Ioannis
Patras (Queen Mary,
Traditionally, the methodologies that have dominated the Face and
Gesture Recognition conference rely primarily on visual information. In this
session, we explore how neurological signals, and in particular EEG, MEG and EMG measurements of brain and muscle activity, can be
utilized for the analysis of facial and body expressions and gestures. The
presentations will cover the analysis of neurological signals that are produced both during the generation/production and during
the perception of facial expressions and gestures. This will include popular
(for the FG audience) topics such as the analysis of
emotional facial expressions and body gestures, and the recognition of
spontaneous vs posed facial expressions based on EEG and
facial EMG. Among the topics that are discussed are
MEG measures of facial expressions, emotions and empathy, EMG
and EEG response patterns to exposure of facial expressions (facial mimicry).
The audience will be introduced to the challenges of
neurological signal analysis, but in most cases will be familiar with the
signal processing and pattern recognition methodologies that are employed.
During a panel session, discussion will be encouraged on
multidisciplinary approaches for facial and body gesture analysis and on
possible applications that can benefit from a multimodal analysis (and in
particular in vision-based and neurological signal - based analysis).
Invited Presentations
Emotional Contagion for Unseen Bodily Expressions: Evidence from Facial EMG
Marco Tamietto1,2 and Beatrice de
Gelder1
1Cognitive and Affective Neuroscience Laboratory,
2Department of Psychology,
Inverse mapping
the neuronal correlates of facial expression processing
Philippe Schyns,
Lucy S. Petro,
Marie L. Smith,
Investigating the production of facial
expressions: an electroencephalographic (EEG) and electromyographic
(EMG) approach
Sebastian Korb, Swiss Center
for Affective Sciences and Department of Psychology, University of Geneva,
Switzerland, sebastian.korb@pse.unige.ch
Didier Grandjean,
Klaus.Scherer,
Training Computer Vision Systems with
Implicit Brain Processing
Ashish Kapoor,
Microsoft Research,
Desney Tan, Microsoft Research,
Pradeep Shenoy,
Eric Horvitz, Microsoft Research,
Panel Discussion
Moderated
by the organizers, the three speakers, the organizers and the audience will
discuss the research presented in this special session and in particular, how
EEG and EMG research approaches can add to existing
knowledge about (audio-visual) detection of affective states from prosody, facial expressions and body postures.
Paper Abstracts and Authors Information
Emotional Contagion for Unseen Bodily Expressions: Evidence from Facial EMG
Marco Tamietto1,2 and Beatrice de
Gelder1
1Cognitive and Affective Neuroscience Laboratory,
2Department of Psychology,
Emotional contagion refers to the tendency to
automatically mimic and synchronize our facial expressions with those of
another person. Recent EMG studies have shown that emotionally-congruent expressive reactions in the observer’s
face may be also elicited by the perception of bodily expressions, thus
challenging the view that emotional contagion is simply due to motor imitation
based on conscious visual recognition. Here we investigated whether emotional
contagion may be triggered by bodily expressions that
cannot be consciously perceived. Facial EMG was recorded in response to the presentation of backwardly
masked happy and fearful bodily expressions. The subjects reacted with
emotionally congruent facial expressions (i.e., greate
zygomaticus major activity for happy expressions, and
greater corrugator supercilli
activity for fearful expressions), despite the fact that they were unable to consciously detect the triggering body stimuli. The
present findings suggest that expressive facial reactions may unfold as an
automatic response driven by the activation of emotion specific affect programs
that are independent from conscious visual recognition.
Marco Tamietto is a neuropsychologist and holds a PhD in
neuroscience. His main research interest is on the neuro-functional
bases of non-conscious emotional processing in neurological patients with blindsight and spatial neglect. He has
been recently awarded with a Veni grant from NWO for a research project focusing on the neurobiological
correlates of conscious and non-conscious emotional communication that will be
pursued in collaboration with Bea de Gelder at
Bea de Gelder’s
research group (http://www.beatricedegelder.com/) focuses on cognitive and affective neuroscience of intersensory perception, between different sensory systems,
primarily on the interaction between seeing and hearing and on how emotion and
cognition interact in humans. Behavioural and neurofunctional approaches (ERPs,
fMRI, MEG and TMS) are used
in an integrated fashion.
Inverse mapping
the neuronal correlates of facial expression processing
Philippe Schyns, Lucy
S. Petro and Marie
L. Smith,
The
brain computations that underlie our ability to recognize facial expressions
involve the extraction of relevant information from the faces of our peers, and
allow us to readily respond in an appropriate manner
to the displayed emotion. Here we present recent advances in understanding the
brain processes underlying the categorization of facial expressions of emotion
using classification image techniques to link both the brain dynamics (EEG) and
the behavioural strategies of three observers with
specific facial features.
Philippe Schyns' research deals with applications of reverse
correlation methods to understand the information processing functions of brain
signals (EEG, MEG and fMRI). The current emphasis is on face processing.
Marie L. Smith’s
research interests include the extension of her work into other neuroimaging modalities (e.g. MEG, fMRI),
the application of new analytical techniques (e.g.
Investigating the production of facial
expressions: an electroencephalographic (EEG) and electromyographic
(EMG) approach
Sebastian Korb, Swiss Center
for Affective Sciences and Department of Psychology, University of Geneva,
Switzerland, sebastian.korb@pse.unige.ch
Didier Grandjean,
Klaus.Scherer,
Facial expressions are part of emotional reactions. However, humans can
voluntarily pose a specific emotional expression without having the
corresponding underlying feeling, or voluntarily modify (e.g. reduce or
enhance) their spontaneous expression in reaction to an emotional event. Few
studies have attempted to distinguish these different processes at the level of
the central nervous system (CNS), even though spontaneous and voluntary facial
expressions are long thought to rely upon distinct
neural circuitries. Here, we review the neural bases of spontaneous and
voluntary facial expressions, report the results of a first study assessing the
Bereitschaftspotential (BP) before voluntary smiles,
and outline a combined EEG/EMG approach for
investigating facial expressions at the level of the CNS.
Since September 2006 Sebastian Korb is a Ph.D.
student at the
Training Computer Vision Systems with
Implicit Brain Processing
Ashish Kapoor,
Microsoft Research,
Desney Tan, Microsoft Research,
Pradeep Shenoy,
Eric Horvitz, Microsoft Research,
Computer vision has been a popular tool for researchers in the Face and
Gesture community. However, there is a tremendous disparity between the workings of computer vision algorithms and how a human brain
processes visual information. In our work, we investigate how to exploit
this disparity for vision tasks. Specifically, we focus on human aided
computing, in which we use an electroencephalograph (EEG) device to measure the
implicit cognitive processing that occurs in the brain as users see and process
images. We discuss how the challenging task of categorizing objects, including
faces, from images can benefit by analyzing human brain responses. We describe
our efforts exploring how we can ideally combine the efforts and competencies
of human and machine computation to achieve improved recognition performance
and highlight methodologies that could allow us to use the brain signal to
train better computer vision systems