1. A computer-implemented method for indexing documents comprising: indexing a document written in one or more natural languages, wherein the indexing includes: applying language-specific rules of one of the one or more natural languages to content of the document; and generating a number of tokens from the content of the document that comply with the language-specific rules; determining a success metric based on the generated number of tokens that complied with the language-specific rules; comparing the success metric to a threshold value, wherein the threshold value indicates a predetermined amount of content in the document designating a multi-lingual document; identifying the document as a multi-lingual document based upon the comparing; and in response to identifying the document as a multi-lingual document, queuing the document for multi-lingual indexing.