11. A non-transitory computer readable storage medium storing one or more programs that when executed by one or more processors of a computer system cause the computer system to: receive documents from one or more databases of documents, wherein each received document is associated with a respective query independent score; obtain a document content identifier for each received document, each document content identifier comprising an identifier of a respective document's content; index at least a subset of the received documents to produce an document index that maps terms to documents in the one or more databases of documents; and while performing the indexing, identify respective sets of received documents having the same content identifier; select a single document in each respective set of documents, in accordance with the query independent scores associated with the documents in the respective set of documents, as a representative document for the respective set of received documents; index the representative document; and with respect to each respective set of received documents, include only the representative document in the document index.
13. The non-transitory computer readable medium of claim 11 , wherein the one or more programs further comprising instructions that when executed by the one or more processors cause the computer system to: compute respective scores for respective received documents, including a respective score for each document in a respective set of documents having the same content identifier, and select the representative document for the respective set of documents in accordance with at least the computed scores for the documents in the respective set of documents.