7. A computer-implemented method for classifying a quality of translated segments generated by a machine translation system, the method comprising: generating an estimated difficulty feature score of a supervised machine learning model in translating input text segments based, at least in part, on a number of out-of-vocabulary words in the input text segments; training a machine translation quality classifier stored in a memory to classify the quality of the translated segments utilizing the supervised machine learning model configured with a misclassification cost configured to offset an imbalance between a plurality of classes of training data, the training data comprising one or more feature scores associated with machine translated segments of a target language and correct class labels for the machine translated segments in the target language, the one or more feature scores including the estimated difficulty feature score; and a loss function configured to penalize a misclassification of a lower-quality translated segment as a higher-quality translated segment more greatly than a misclassification of a higher-quality translated segment as a lower-quality translated segment; and utilizing the machine translation quality classifier at a computer in a service provider network to classify the quality of the translated segments generated by the machine translation system into the plurality of classes.
10. The computer-implemented method of claim 7 , further comprising: aggregating a plurality of classifications for translated segments; and generating one or more document-level or corpus-level distribution statistics based upon the aggregated classifications.