Title: Data Analysis in Multimedia Quality Assessment: Revisiting the Statistical Tests,
Abstract: Assessment of multimedia quality relies heavily on subjective assessment, and
is typically done by human subjects in the form of preferences or continuous
ratings. Such data is crucial for analysis of different multimedia processing
algorithms as well as validation of objective (computational) methods for the
said purpose. To that end, statistical testing provides a theoretical framework
towards drawing meaningful inferences, and making well grounded conclusions and
recommendations. While parametric tests (such as t test, ANOVA, and error
estimates like confidence intervals) are popular and widely used in the
community, there appears to be a certain degree of confusion in the application
of such tests. Specifically, the assumption of normality and homogeneity of
variance is often not well understood. Therefore, the main goal of this paper
is to revisit them from a theoretical perspective and in the process provide
useful insights into their practical implications. Experimental results on both
simulated and real data are presented to support the arguments made. A software
implementing the said recommendations is also made publicly available, in order
to achieve the goal of reproducible research.