1. A method of processing documents, comprising: at a computer system comprising one or more processors and memory storing one or more programs for execution by the one or more processors so as to perform the method, grouping a set of documents into a plurality of clusters, the set of documents comprising a sequence of tokens, wherein each cluster includes one or more documents in the set of documents; generating a compressed representation of the set of documents from which respective documents in the set can be reconstructed, including generating, for the plurality of clusters, a compressed sequence of tokens by eliding duplicate sequences of tokens within respective clusters; and generating an index of the compressed sequence of tokens, including indexing each respective token in the compressed sequence of tokens based on a respective token position of the respective token in the compressed sequence.