Title: Learning Pain from Action Unit Combinations: A Weakly Supervised Approach via Multiple Instance Learning,
Abstract: Patient pain can be detected highly reliably from facial expressions using a
set of facial muscle-based action units (AUs) defined by the Facial Action
Coding System (FACS). A key characteristic of facial expression of pain is the
simultaneous occurrence of pain-related AU combinations, whose automated
detection would be highly beneficial for efficient and practical pain
monitoring. Existing general Automated Facial Expression Recognition (AFER)
systems prove inadequate when applied specifically for detecting pain as they
either focus on detecting individual pain-related AUs but not on combinations
or they seek to bypass AU detection by training a binary pain classifier
directly on pain intensity data but are limited by lack of enough labeled data
for satisfactory training. In this paper, we propose a new approach that mimics
the strategy of human coders of decoupling pain detection into two consecutive
tasks: one performed at the individual video-frame level and the other at
video-sequence level. Using state-of-the-art AFER tools to detect single AUs at
the frame level, we propose two novel data structures to encode AU combinations
from single AU scores. Two weakly supervised learning frameworks namely
multiple instance learning (MIL) and multiple clustered instance learning
(MCIL) are employed corresponding to each data structure to learn pain from
video sequences. Experimental results show an 87% pain recognition accuracy
with 0.94 AUC (Area Under Curve) on the UNBC-McMaster Shoulder Pain Expression
dataset. Tests on long videos in a lung cancer patient video dataset
demonstrates the potential value of the proposed system for pain monitoring in
clinical settings.