10. The method of claim 1 wherein identifying one or more languages as being languages of the document includes: determining whether each segment t is a long segment or a short segment; for each segment t that is a long segment: identifying the language Lt that has the best segment score St(L); determining whether the segment score St(Lt) for the language Lt satisfies a threshold criterion; and identifying the language Lt as a language of the document in the event that the segment score St(Lt) for the language Lt satisfies the threshold criterion; and for each segment t that is a short segment: determining whether any one or more other short segments k have the same active subset (At) as the segment t; for each other short segment k that has the same active subset At as the segment t, aggregating the other segment score S k (L) with the segment score St(L) for each language L in active subset At to determine an aggregate score for language L; identifying the language Lt that has the best aggregate score; determining whether the aggregate score satisfies a threshold criterion; and identifying the language Lt as a language of the document in the event that the aggregate score for the language Lt satisfies the threshold criterion.
1. A method of identifying one or more languages for a document, the languages being selected from a set of candidate languages, the method comprising: dividing the set of candidate languages into a plurality of disjoint subsets, wherein any two languages that are in different disjoint subsets do not overlap with each other; segmenting the document into one or more segments (t) of consecutive characters, wherein each segment t contains n-grams that have greater than a default probability of occurrence only for languages in an active one of the disjoint subsets (A t ); for each segment t, generating a segment score (S t (L)) for each language (L) in the active one of the disjoint subsets A t ; identifying, by a processor, one or more languages as being languages of the document based on the segment scores S t (L) for all of the segments t and languages L; and storing, in a computer readable storage device, information indicating the one or more languages of the document.