title: Python regex - split alphanumeric characters but remove and combine hyphenated words
body: <p>I am parsing over two million text files for a project and need your help with regex <strong>make it faster</strong>. I need to split the words so I am able to do it using:</p>
  
  <pre><code>text_file = re.sub(r'\W+',' ', text_file)
  </code></pre>
  
  <p>However, for hyphenated words, I would like to remove the hyphen but <strong>combine</strong> the word (i.e. non-profit would be nonprofit instead of "non", "profit"). Currently, I am able to do this with two search and replace (not ideal at all). The end output would be a list of words that will used by collections.Counter() object.</p>
  
  <pre><code>def mod1(text_file):
       text_file = re.sub('\s*-\s*', '', text_file) # handling hyphens :( O(n)
       text_file = re.sub(r'\W+',' ', text_file)
       text_file = text_file.lower()
       return(text_file)
  </code></pre>
  
  <p>Any <strong>faster processing</strong> tips to do this beyond regex would also be much appreciated. Thanks.</p>