Human activity recognition is one of the important research topics in computer vision and video understanding. It is often assumed that high quality video sequences are available for recognition. However, relaxing such a requirement and implementing robust recognition using videos having reduced data rates can achieve efficiency in storing and transmitting video data. Three-dimensional video scalability, which refers to the possibility of reducing spatial, temporal, and quality resolutions of videos, is an effective way for flexible representation and management of video data. In this paper, we investigate the impact of the video scalability on multi-view activity recognition. We employ both a spatiotemporal feature extraction-based method and a deep learning-based method using convolutional and recurrent neural networks. The recognition performance of the two methods is examined, along with in-depth analysis regarding how their performance vary with respect to various scalability combinations. In particular, we demonstrate that the deep learning-based method can achieve significantly improved robustness in comparison to the feature-based method. Furthermore, we investigate optimal scalability combinations with respect to bitrate in order to provide useful guidelines for an optimal operation policy in resource-constrained activity recognition systems.
@cite_14: Unmanned aerial systems (UAS) equipped with electro-optic (EO) full motion video (FMV) sensors often need to transmit image sequences over a limited communications channel, requiring either intense compression, reduced frame rate, or reduced resolution to reach the receiver. In an attempt to improve rate-distortion performance of common video compression algorithms, such as H.264 AVC, several groups are developing compres- sion methods to improve video quality at low bitrates. Concepts of these next generation methods, including H.265 HEVC, Google’s VP9, and Xiph.org’s Daala are examined in contrast to H.264 AVC, BBC’s Dirac, and Motion-JPEG2000 within the context of aerial surveillance. We present a compression performance analysis of these algorithms according to PSNR.
@cite_22: This paper investigates the robustness of two state-of-theart action recognition algorithms: a pixel domain approach based on 3D convolutional neural networks (C3D) and a compressed domain approach requiring only partial decoding of the video, based on feature description using motion vectors and Fisher vector encoding (MV-FV). We study the robustness of the two algorithms against: (i) quality variations, (ii) changes in video encoding scheme, (iii) changes in resolutions. Experiments are performed on the HMDB51 dataset. Our main findings are that C3D is robust to variations of these parameters while the MV-FV is very sensitive. Hence, we consider C3D as a baseline method for our analysis. We also analyze the reasons behind these different behaviors and discuss their practical implications.
@cite_1: Human action recognition from videos is very important for visual analytics. Due to increasing abundance of diverse video content in the era of big data, research on human action recognition has recently shifted towards more challenging and realistic settings. Frame rate is one of key issues in diverse and realistic video settings. While there have been several evaluation studies investigating different aspects of action recognition such as different visual descriptors, the frame rate issue has been seldom addressed in the literature. Therefore, in this paper, we investigate the impact of frame rate on human action recognition with several state-of-the-art approaches and three benchmark datasets. Our experimental results indicate that those state-of-the-art approaches are not robust to the variations of frame rate. As a result, more robust visual features and advanced learning algorithms are required to further improve human action recognition performance towards its more practical deployments. In addition, we investigate key-frame selection techniques for choosing a set of suitable frames from an action sequence for action recognition. Promising results indicate that well designed key-frame selection methods can produce a set of representative frames and eventually reduce the impact of frame rate on the performance of human action recognition. HighlightsOne of the first studies on evaluating the impact of frame rate on video based human action recognition.Evaluation with four state-of-the-art methods and three widely used datasets.Investigation on novel key frame selection method for action recognition.A comprehensive study of exploring the impact of frame rate on action recognition performance.
@cite_6: Most studies in the literature for video quality assessment have been focused on the evaluation of quantized video sequences at fixed and high spatial and temporal resolutions. Only limited work has been reported for assessing video quality under different spatial and temporal resolutions. In this paper, we consider a wider scope of video quality assessment in the sense of considering multiple dimensions. In particular, we address the problem of evaluating perceptual visual quality of low bit-rate videos under different settings and requirements. Extensive subjective view tests for assessing the perceptual quality of low bit-rate videos have been conducted, which cover 150 test scenarios and include five distinctive dimensions: encoder type, video content, bit rate, frame size, and frame rate. Based on the obtained subjective testing results, we perform thorough statistical analysis to study the influence of different dimensions on the perceptual quality and some interesting observations are pointed out. We believe such a study brings new knowledge into the topic of cross-dimensional video quality assessment and it has immediate applications in perceptual video adaptation for scalable video over mobile networks.
@cite_3: 
@cite_0: Objective metrics are evaluated on quality assessment of multidimensional video scalability.Both generic image video metrics and scalability-targeting metrics are included.Evaluation is systematically performed on three different data sets.In-depth analysis in terms of performance and complexity of metrics is provided.Overall, scalability-targeting metrics show best performance. Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.
@cite_2: 
@cite_13: Scalability is a powerful concept for adaptive video content delivery to many end users having heterogeneous and dynamic characteristics of networks and devices. In order to maximize users' quality of experience by selecting appropriate combinations of multiple scalability parameters, it is crucial to understand and model the relationship between multidimensional scalability and perceived quality. In this article, we address the latest advances in subjective and objective quality evaluation of multidimensional video scalability for optimal content distribution, present their applications, and discuss future trends and challenges.
@cite_17: We assess the impact of the H.264 video codec on the match performance of automated face recognition in surveillance and mobile video applications. A set of two hundred access control (90 pixel inter-pupilary distance) and distance surveillance (45 pixel inter-pupilary distance) videos taken under non-ideal imaging and facial recognition (e.g., pose, illumination, and expression) conditions were matched using two commercial face recognition engines in the studies. The first study evaluated automated face recognition performance on access control and distance surveillance videos at CIF and VGA resolutions using the H.264 baseline profile at nine bitrates rates ranging from 8kbs to 2048kbs. In our experiments, video signals were able to be compressed up to 128kbs before a significant drop face recognition performance occurred. The second study evaluated automated face recognition on mobile devices at QCIF, iPhone, and Android resolutions for each of the H.264 PDA profiles. Rank one match performance, cumulative match scores, and failure to enroll rates are reported.
In general, videos can have various combinations of spatial, temporal, and quality resolutions, which is noted as video scalability. Previous studies in several fields investigated how it affects the performance of the video systems, including surveillance @cite_17 @cite_14 , face recognition @cite_3 @cite_17 , visual perception of users @cite_6 @cite_13 @cite_0 , and so on. The video scalability has been also considered in human activity recognition tasks. Harjanto investigated the impact of different frame rates on human activity recognition in several feature extraction-based approaches, and observed degradation of recognition performance for reduced frame rates @cite_1 . See and Rahman analyzed the impact of video scalability on activity recognition using extremely low quality videos @cite_2 . Srinivasan analyzed the performance of a method using compression-domain features (i.e., motion vectors), and confirmed the fluctuation of performance with respect to spatial and quality variations @cite_22 . However, these studies focused only on feature extraction-based methods and furthermore, did not consider various combinations of the video scalability dimensions thoroughly.