12. The non-transitory computer readable medium of claim 11 , wherein the document content identifier for a respective document comprises a fixed length fingerprint of document content of the respective document.
11. A non-transitory computer readable storage medium storing one or more programs that when executed by one or more processors of a computer system cause the computer system to: receive documents from one or more databases of documents, wherein each received document is associated with a respective query independent score; obtain a document content identifier for each received document, each document content identifier comprising an identifier of a respective document's content; index at least a subset of the received documents to produce an document index that maps terms to documents in the one or more databases of documents; and while performing the indexing, identify respective sets of received documents having the same content identifier; select a single document in each respective set of documents, in accordance with the query independent scores associated with the documents in the respective set of documents, as a representative document for the respective set of received documents; index the representative document; and with respect to each respective set of received documents, include only the representative document in the document index.