Title: Fisher consistency for prior probability shift,
Abstract: We introduce Fisher consistency in the sense of unbiasedness as a desirable
property for estimators of class prior probabilities. Lack of Fisher
consistency could be used as a criterion to dismiss estimators that are
unlikely to deliver precise estimates in test datasets under prior probability
and more general dataset shift. The usefulness of this unbiasedness concept is
demonstrated with three examples of classifiers used for quantification:
Adjusted Classify & Count, EM-algorithm and CDE-Iterate. We find that Adjusted
Classify & Count and EM-algorithm are Fisher consistent. A counter-example
shows that CDE-Iterate is not Fisher consistent and, therefore, cannot be
trusted to deliver reliable estimates of class probabilities.