{'fraction_non_alphanumeric': 0.015895953757225433, 'fraction_numerical': 0.0, 'mean_word_length': 5.663461538461538, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An algorithm for automatic words extraction from a stream of phones in dictionary-based large vocabulary continuous speech recognition systems

As improvements on acoustic modeling have rapidly progressed in recent years thanks to the impressive gains in performance obtained using deep neural networks (DNNs), language modeling remains a bottleneck for high performance large vocabulary continuous speech recognition (LVCSR) systems. In this paper an algorithm for automatic words extraction from a stream of phones is suggested to be used in a dictionary-based LVCSR system, to overcome the limitations of current LVCSR systems. Experimental results show the effectiveness of this approach.