8. A computer system for improving indexing throughput, the computer system comprising: a document repository to store one or more documents; a processor; at least one memory and storage in communication with the processor, storing: a repository crawler in a content extraction engine executable on the processor to receive one or more documents from the document repository; a categorizer in the content extraction engine executable on the processor to categorize the one or more documents based on document type, document size and processing priority, and to assign buckets to the categorized one or more documents; the processor calculating a document type priority for the document type of the one or more documents in the assigned buckets, wherein the document type priority is calculated as a function of an extraction efficiency for extracting content from the one or more documents for the corresponding document type; a scheduler in the content extraction engine executable on the processor to schedule the categorized one or more documents for indexing according to a bucket type, the calculated document type priority and number of threads available to process the bucket; a content extractor in the content extraction engine executable on the processor to extract content from the categorized one or more documents; an indexing engine to index the categorized one or more documents; and an index to store the one or more indexed documents.
8. A computer system for improving indexing throughput, the computer system comprising: a document repository to store one or more documents; a processor; at least one memory and storage in communication with the processor, storing: a repository crawler in a content extraction engine executable on the processor to receive one or more documents from the document repository; a categorizer in the content extraction engine executable on the processor to categorize the one or more documents based on document type, document size and processing priority, and to assign buckets to the categorized one or more documents; the processor calculating a document type priority for the document type of the one or more documents in the assigned buckets, wherein the document type priority is calculated as a function of an extraction efficiency for extracting content from the one or more documents for the corresponding document type; a scheduler in the content extraction engine executable on the processor to schedule the categorized one or more documents for indexing according to a bucket type, the calculated document type priority and number of threads available to process the bucket; a content extractor in the content extraction engine executable on the processor to extract content from the categorized one or more documents; an indexing engine to index the categorized one or more documents; and an index to store the one or more indexed documents. 11. The computer system of claim 8 , wherein the content extractor comprises threads for extracting content from the one or more documents.