In this paper, we present a novel method of no-reference image quality assessment (NR-IQA), which is to predict the perceptual quality score of a given image without using any reference image. The proposed method harnesses three functions (i) the visual attention mechanism, which affects many aspects of visual perception including image quality assessment, however, is overlooked in the NR-IQA literature. The method assumes that the fixation areas on an image contain key information to the process of IQA. (ii) the robust averaging strategy, which is a means --- supported by psychology studies --- to integrating multiple step-wise evidence to make a final perceptual judgment. (iii) the multi-task learning, which is believed to be an effectual means to shape representation learning and could result in a more generalized model. To exploit the synergy of the three, we consider the NR-IQA as a dynamic perception process, in which the model samples a sequence of "informative" areas and aggregates the information to learn a representation for the tasks of jointly predicting the image quality score and the distortion type. The model learning is implemented by a reinforcement strategy, in which the rewards of both tasks guide the learning of the optimal sampling policy to acquire the "task-informative" image regions so that the predictions can be made accurately and efficiently (in terms of the sampling steps). The reinforcement learning is realized by a deep network with the policy gradient method and trained through back-propagation. In experiments, the model is tested on the TID2008 dataset and it outperforms several state-of-the-art methods. Furthermore, the model is very efficient in the sense that a small number of fixations are used in NR-IQA.
@cite_9: Since the human visual system (HVS) is the ultimate assessor of image quality, current research on the design of objective image quality metrics tends to include an important feature of the HVS, namely, visual attention. Different metrics for image quality prediction have been extended with a computational model of visual attention, but the resulting gain in reliability of the metrics so far was variable. To better understand the basic added value of including visual attention in the design of objective metrics, we used measured data of visual attention. To this end, we performed two eye-tracking experiments: one with a free-looking task and one with a quality assessment task. In the first experiment, 20 observers looked freely to 29 unimpaired original images, yielding us so-called natural scene saliency (NSS). In the second experiment, 20 different observers assessed the quality of distorted versions of the original images. The resulting saliency maps showed some differences with the NSS, and therefore, we applied both types of saliency to four different objective metrics predicting the quality of JPEG compressed images. For both types of saliency the performance gain of the metrics improved, but to a larger extent when adding the NSS. As a consequence, we further integrated NSS in several state-of-the-art quality metrics, including three full-reference metrics and two no-reference metrics, and evaluated their prediction performance for a larger set of distortions. By doing so, we evaluated whether and to what extent the addition of NSS is beneficial to objective quality prediction in general terms. In addition, we address some practical issues in the design of an attention-based metric. The eye-tracking data are made available to the research community .
@cite_3: Automatic image quality assessment (IQA) attempts to use computational models to measure the image quality in consistency with subjective ratings. In the past decades, dozens of IQA models have been proposed. Though some of them can predict subjective image quality accurately, their computational costs are usually very high. To meet real-time requirements, in this paper, we propose a novel fast and effective IQA index, namely spectral residual based similarity (SR-SIM), based on a specific visual saliency model, spectral residual visual saliency. SR-SIM is designed based on the hypothesis that an image's visual saliency map is closely related to its perceived quality. Extensive experiments conducted on three large-scale IQA datasets indicate that SR-SIM could achieve better prediction performance than the other state-of-the-art IQA indices evaluated. Moreover, SR-SIM can have a quite low computational complexity. The Matlab source code of SR-SIM and the evaluation results are available online at http: sse.tongji.edu.cn linzhang IQA SR-SIM SR-SIM.htm.
@cite_24: Image quality assessment (IQA) has thrived for decades, and researchers continue to explore how the human brain perceives visual stimuli. Psychological evidence shows that humans prefer qualitative descriptions when evaluating image quality, yet most researches still concentrate on numerical descriptions. Furthermore, handcrafting features are widely used in this community, which constrains the models' flexibility. A novel model is proposed with two major advantages: the saliency-guided feature learning can learn features unsupervisedly, and the deep framework recasts IQA as a classification problem, analogous to human qualitative evaluation. Experiments validate the proposed model's effectiveness.
@cite_23: Image quality assessment (IQA) tries to estimate human perception based image visual quality in an objective manner. Existing approaches target this problem with or without reference images. For no-reference image quality assessment, there is no given reference image or any knowledge of the distortion type of the image. Previous approaches measure the image quality from signal level rather than semantic analysis. They typically depend on various features to represent local characteristic of an image. In this paper we propose a new no-reference (NR) image quality assessment (IQA) framework based on semantic obviousness. We discover that semantic-level factors affect human perception of image quality. With such observation, we explore semantic obviousness as a metric to perceive objects of an image. We propose to extract two types of features, one to measure the semantic obviousness of the image and the other to discover local characteristic. Then the two kinds of features are combined for image quality estimation. The principles proposed in our approach can also be incorporated with many existing IQA algorithms to boost their performance. We evaluate our approach on the LIVE dataset. Our approach is demonstrated to be superior to the existing NR-IQA algorithms and comparable to the state-of-the-art full-reference IQA (FR-IQA) methods. Cross-dataset experiments show the generalization ability of our approach.
@cite_2: 
Objectness and Saliency in IQA: Although there lacks of literature that organically fuses the attentional mechanism into the NR-IQA, the semantic objectness or saliency has been applied. Objectness and saliency are static property of image regions, whereas attention is an active perception process of an observer. Liu al @cite_9 determine the final score of an image by averaging the predicted patch scores with weights. The patch weights are the saliency values obtained from eye-tracking data. The performance gain of the method justifies the importance of introducing visual attention to IQA. Zhang and Li @cite_3 argue that visual saliency and perceptual quality are highly related, and they utilize the relationship between a reference image saliency map and its distortion image saliency map to predict image quality scores. Zhang al @cite_23 propose an IQA algorithm using object-like regions. They assume that semantic regions contribute to perceptual quality assessment. Hou and Gao @cite_24 propose a saliency-guided framework whose idea is similar to @cite_23 . In summary, these methods exploit image saliency maps in post-processing, , adopt saliency-weighted average score rather than a uniform average score as final prediction. Zhang al @cite_2 study different combinations of different saliency models and IQA methods.