1. A method, in a data processing system, for classifying media data, comprising: responsive to receiving input media data, classifying the input media data for later access based on classification; applying a plurality of classifiers in a hierarchical classification structure to the input media data; generating confidence scores for the plurality of classifiers in the hierarchical classification structure; modifying a confidence score automatically for a classifier in the hierarchical classification structure based on a relationship between the classifier and other classifiers in the hierarchical classification structure and the confidence scores of the other classifiers in the hierarchical classification structure by applying a confusion factor to the confidence score based on a correspondence between the confidence score and confidence scores of mutually exclusive classifiers in the hierarchical classification structure; generating a representation of the input media data based on the modified confidence score being higher than a predetermined threshold confidence score; and storing the generated representation of the input media data.