how to extract high frequency phrase
This uses python and NLTK

import nltk

text = """abcdef
abcd
xabcd
b
bcef
ef
aef
"""

def tokens(lines):
    for line in lines:
        for n in range(1, len(line)):
            ts = nltk.ngrams(line, n)
            for t in ts: yield str.join('', t) 
        yield line

ts = tokens(text.splitlines())
freq = nltk.FreqDist(ts)
print(freq.most_common())
# [('b', 5), ('c', 4), ('ef', 4), ('bc', 4), ('e', 4)...


There are other solutions, of course, but I think this meets your criteria. You may need to optimize if your lines are huge as this will create every possible n-gram.