1. A method, in a data processing system, for classifying media data for later access based on the classification, comprising: receiving input media data; applying a plurality of classifiers in a hierarchical classification structure to the input media data; generating confidence scores for the plurality of classifiers in the hierarchical classification structure; modifying a confidence score for a classifier in the hierarchical classification structure based on a relationship between the classifier and other classifiers in the hierarchical classification structure and the confidence scores of the other classifiers in the hierarchical classification structure; and generating a representation of the input media data based on the modified confidence score.