Emotion recognition from facial expressions is tremendously useful, especially when coupled with smart devices and wireless multimedia applications. However, the inadequate network bandwidth often limits the spatial resolution of the transmitted video, which will heavily degrade the recognition reliability. We develop a novel framework to achieve robust emotion recognition from low bit rate video. While video frames are downsampled at the encoder side, the decoder is embedded with a deep network model for joint super-resolution (SR) and recognition. Notably, we propose a novel max-mix training strategy, leading to a single "One-for-All" model that is remarkably robust to a vast range of downsampling factors. That makes our framework well adapted for the varied bandwidths in real transmission scenarios, without hampering scalability or efficiency. The proposed framework is evaluated on the AVEC 2016 benchmark, and demonstrates significantly improved stand-alone recognition performance, as well as rate-distortion (R-D) performance, than either directly recognizing from LR frames, or separating SR and recognition.
@cite_30: 
@cite_32: Most automatic facial expression analysis (AFEA) systems attempt to recognize facial expressions from data collected in a highly controlled environment with very high resolution frontal faces ( face regions greater than 200 x 200 pixels). However, in real environments, the face image is often in lower resolution and with head motion. It is unclear that the performance of AFEA systems for low resolution face images. The general approach to AFEA consists of 3 steps: face acquisition, facial feature extraction, and facial expression recognition. This paper explores the effects of different image resolutions for each step of facial expression analysis. The different approaches are compared for face detection, face data extraction and expression recognition. A total of five different resolutions of the head region are studied (288x384, 144x192, 72x96, 36x48, and 18Xx24) based on a widely used public database. The lower resolution images are down-sampled from the originals.
@cite_3: 
@cite_0: This paper presents a meta-analysis for covariates that affect performance of face recognition algorithms. Our review of the literature found six covariates for which multiple studies reported effects on face recognition performance. These are: age of the person, elapsed time between images, gender of the person, the person's expression, the resolution of the face images, and the race of the person. The results presented are drawn from 25 studies conducted over the past 12 years. There is near complete agreement between all of the studies that older people are easier to recognize than younger people, and recognition performance begins to degrade when images are taken more than a year apart. While individual studies find men or women easier to recognize, there is no consistent gender effect. There is universal agreement that changing expression hurts recognition performance. If forced to compare different expressions, there is still insufficient evidence to conclude that any particular expression is better than another. Higher resolution images improve performance for many modern algorithms. Finally, given the studies summarized here, no clear conclusions can be drawn about whether one racial group is harder or easier to recognize than another.
@cite_24: Automatic facial expression analysis is an interesting and challenging problem, and impacts important applications in many areas such as human-computer interaction and data-driven animation. Deriving an effective facial representation from original face images is a vital step for successful facial expression recognition. In this paper, we empirically evaluate facial representation based on statistical local features, Local Binary Patterns, for person-independent facial expression recognition. Different machine learning methods are systematically examined on several databases. Extensive experiments illustrate that LBP features are effective and efficient for facial expression recognition. We further formulate Boosted-LBP to extract the most discriminant LBP features, and the best recognition performance is obtained by using Support Vector Machine classifiers with Boosted-LBP features. Moreover, we investigate LBP features for low-resolution facial expression recognition, which is a critical problem but seldom addressed in the existing work. We observe in our experiments that LBP features perform stably and robustly over a useful range of low resolutions of face images, and yield promising performance in compressed low-resolution video sequences captured in real-world environments.
@cite_23: Deep learning has been successfully applied to image super resolution (SR). In this paper, we propose a deep joint super resolution (DJSR) model to exploit both external and self similarities for SR. A Stacked Denoising Convolutional Auto Encoder (SDCAE) is first pre-trained on external examples with proper data augmentations. It is then fine-tuned with multi-scale self examples from each input, where the reliability of self examples is explicitly taken into account. We also enhance the model performance by sub-model training and selection. The DJSR model is extensively evaluated and compared with state-of-the-arts, and show noticeable performance improvements both quantitatively and perceptually on a wide range of images.
Empirical studies @cite_0 @cite_3 in face recognition proved that a minimum face resolution between @math and @math is required for most stand-alone recognition algorithms, whose performance would be much degraded when applied with even lower resolutions @cite_23 @cite_30 . In the emotion recognition literature, most existing methods assumed the availability of HR frontal faces. @cite_32 first investigated the effects of different image resolutions for facial expression analysis. The author concluded that while the performance difference was negligible when the head region resolution was @math or higher, the recognition turned growingly unreliable when head region resolution was lower than @math . It is thus desirable to obtain more robust features for LR images and low-intensity expressions @cite_24