14. An apparatus for analyzing, on shared and distributed computing systems, information comprising one or more documents, the apparatus comprising: at least one hardware memory and at least one hardware processor; distributing one or more distinct sets of the documents to each of a plurality of processors, wherein each processor of the plurality of the processors has a distinct set of document, wherein each processor of the plurality of processors executes a process that performs operations on the distinct set of the document of the one or more distinct sets of the documents substantially in parallel with other processors of the plurality of processors; for each of the plurality of processors, performing an operation comprising: a) computing term statistic for terms contained in each of the processor's distinct set of documents, and generating a local set of statistics for terms within the processor's distinct set of documents; b) contributing in a collective operation performed by the processor the local set of statistics to a global set of term statistics, wherein the global set of term statistics comprises an aggregate of the local set of statistics from each and every processor of the plurality of processors; c) distributing the global set of term statistics to each and every processor of the plurality of processors; d) receiving a predetermined set of vocabulary from a global vocabulary; e) filtering the terms in the predetermined set of vocabulary based on a statistics-based protocol utilizing the global set of term statistics, including ranking the predetermined set of vocabulary according to a predetermined criteria and excluding the terms based on a predetermined threshold; f) merging the terms to a global topicality list; and g) ranking and truncating the global topicality list to form a major term set, wherein the major term set is broadcasted to each and every processor of the plurality of processors.
15. The apparatus as recited in claim 14 , wherein said filtering comprises computing topicality rankings for the terms in the assigned portion of the global vocabulary, sorting the ranked terms into lists according to predetermined criteria, and truncating the lists to exclude terms beyond a predetermined threshold.