Title: High-Level Concepts for Affective Understanding of Images,
Abstract: This paper aims to bridge the affective gap between image content and the
emotional response of the viewer it elicits by using High-Level Concepts
(HLCs). In contrast to previous work that relied solely on low-level features
or used convolutional neural network (CNN) as a black-box, we use HLCs
generated by pretrained CNNs in an explicit way to investigate the
relations/associations between these HLCs and a (small) set of Ekman's
emotional classes. As a proof-of-concept, we first propose a linear admixture
model for modeling these relations, and the resulting computational framework
allows us to determine the associations between each emotion class and certain
HLCs (objects and places). This linear model is further extended to a nonlinear
model using support vector regression (SVR) that aims to predict the viewer's
emotional response using both low-level image features and HLCs extracted from
images. These class-specific regressors are then assembled into a regressor
ensemble that provide a flexible and effective predictor for predicting
viewer's emotional responses from images. Experimental results have
demonstrated that our results are comparable to existing methods, with a clear
view of the association between HLCs and emotional classes that is ostensibly
missing in most existing work.