Efficient database model for storing data indexed by n-grams I'm working on an application which requires creating a very large database of n-grams that exist in a large text corpus.

I need three efficient operation types: Lookup and insertion indexed by the n-gram itself, and querying for all n-grams that contain a sub-n-gram.

This sounds to me like the database should be a gigantic document tree, and document databases, e.g. Mongo, should be able to do the job well, but I've never used those at scale.

Knowing the Stack Exchange question format, I'd like to clarify that I'm not asking for suggestions on specific technologies, but rather a type of database that I should be looking for to implement something like this at scale.
See Lucene NGramTokenizer

Are you sure you can't just use lucene or similar indexing techniques?

Inverted indexes will store the n-gram only once, then just the document ids that contain the ngram; they don't store this as highly redundant raw text.

As for finding ngrams that contain your query sub-n-gram, I would build an index on the observed ngrams, e.g. using a second lucene index, or any other substring index such as a trie or suffix tree. If your data is dynamic, probably lucene is a reasonable choice, using phrase queries to find your n-grams.