I have a collection of strings which have a lot of common substrings,and I'm trying to find a good way to define tokens to compress them.For instance, if my strings are:s1 = Strings2 = Bools3 = String -> Bools4 = String -> Strings5 = (String -> String) -> String -> [Bool]then I might want to use the tokens:$1 = String$2 = Bool$3 = $1 -> $1so that the strings may be defined as:s1 = $1s2 = $2s3 = $1 -> $2s4 = $3s5 = ($3) -> $1 -> [$2](In fact, now it's clear that the definition $4 =  ->  might be good one to add.)I'm looking for a good (perhaps the best?) way to choose thetoken definitions. I'm interested in minimizing the total lengthof the token definitions + the resulting string definitions.Any ideas?UpdateIt's kinda related to this SO question: Huffman encoding with variable length symbols
finding optimal token definitions for compression
algorithms;strings;compression;huffman encoding