Title: Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data,
Abstract: Recently, two new indicators (Equalized Mean-based Normalized Proportion
Cited, EMNPC; Mean-based Normalized Proportion Cited, MNPC) were proposed which
are intended for sparse scientometrics data. The indicators compare the
proportion of mentioned papers (e.g. on Facebook) of a unit (e.g., a researcher
or institution) with the proportion of mentioned papers in the corresponding
fields and publication years (the expected values). In this study, we propose a
third indicator (Mantel-Haenszel quotient, MHq) belonging to the same indicator
family. The MHq is based on the MH analysis - an established method in
statistics for the comparison of proportions. We test (using citations and
assessments by peers, i.e. F1000Prime recommendations) if the three indicators
can distinguish between different quality levels as defined on the basis of the
assessments by peers. Thus, we test their convergent validity. We find that the
indicator MHq is able to distinguish between the quality levels in most cases
while MNPC and EMNPC are not. Since the MHq is shown in this study to be a
valid indicator, we apply it to six types of zero-inflated altmetrics data and
test whether different altmetrics sources are related to quality. The results
for the various altmetrics demonstrate that the relationship between altmetrics
(Wikipedia, Facebook, blogs, and news data) and assessments by peers is not as
strong as the relationship between citations and assessments by peers.
Actually, the relationship between citations and peer assessments is about two
to three times stronger than the association between altmetrics and assessments
by peers.