title: how to remove everything but letters, numbers and ! ? . ; , @ ' using regex in python pandas df?
body: <p>I am trying to remove everythin but letters, numbers and ! ? . ; , @ ' from my python pandas column text.
  I have already read some other questions on the topic, but still can not make mine work. </p>
  
  <p>Here is an example of what I am doing:</p>
  
  <pre><code>import pandas as pd
  df = pd.DataFrame({'id':[1,2,3,4],
                    'text':['hey+ guys! wuzup',
                                'hello p3ople!What\'s up?',
                                'hey, how-  thing == do##n',
                                'my name is bond, james b0nd']}
                  )
  </code></pre>
  
  <p>Then we have the following table:</p>
  
  <pre><code>id                         text
  1              hey+ guys! wuzup
  2      hello p3ople!What\'s up?
  3     hey, how-  thing == do##n
  4   my name is bond, james b0nd
  </code></pre>
  
  <p>Now, tryng to remove everything but letters, numbers and ! ? . ; , @ '</p>
  
  <p>First try:</p>
  
  <pre><code>df.loc[:,'text'] = df['text'].str.replace(r"^(?!(([a-zA-z]|[\!\?\.\;\,\@\'\"]|\d))+)$",' ',regex=True)
  </code></pre>
  
  <p>output</p>
  
  <pre><code>id                         text
  1              hey+ guys! wuzup
  2       hello p3ople!What's up?
  3      hey, how- thing == do##n
  4   my name is bond, james b0nd
  </code></pre>
  
  <p>Second try</p>
  
  <pre><code>df.loc[:,'text'] = df['text'].str.replace(r"(?i)\b(?:(([a-zA-Z\!\?\.\;\,\@\'\"\:\d])))",' ',regex=True)
  </code></pre>
  
  <p>output</p>
  
  <pre><code>id                         text
  1                  ey+ uys uzup
  2              ello 3ople hat p
  3            ey ow- hing == o##
  4          y ame s ond ames 0nd
  </code></pre>
  
  <p>Third try</p>
  
  <pre><code>df.loc[:,'text'] = df['text'].str.replace(r'(?i)(?&lt;!\w)(?:[a-zA-Z\!\?\.\;\,\@\'\"\:\d])',' ',regex=True)
  </code></pre>
  
  <p>output</p>
  
  <pre><code>id                         text
  1                 ey+ uys! uzup
  2           ello 3ople! hat' p?
  3           ey, ow- hing == o##
  4         y ame s ond, ames 0nd
  </code></pre>
  
  <p>Afterwars, I also tried using re.sub() function using the same regex patterns, but still did not manage to have the expected the result. Being this expected result as follows:</p>
  
  <pre><code>id                         text
  1               hey guys! wuzup
  2       hello p3ople!What's up?
  3          hey, how-  thing don
  4   my name is bond, james b0nd
  </code></pre>
  
  <p>Can anyone help me with that?</p>
  
  <p>Links that I have seen over the topic:</p>
  
  <p><a href="https://stackoverflow.com/questions/31364852/is-there-a-way-to-remove-everything-except-characters-numbers-and-from-a-st">Is there a way to remove everything except characters, numbers and &#39;-&#39; from a string</a></p>
  
  <p><a href="https://stackoverflow.com/questions/59570950/how-do-check-if-a-text-column-in-my-dataframe-contains-a-list-of-possible-patte">How do check if a text column in my dataframe, contains a list of possible patterns, allowing mistyping?</a></p>
  
  <p><a href="https://stackoverflow.com/questions/44227748/removing-newlines-from-messy-strings-in-pandas-dataframe-cells">removing newlines from messy strings in pandas dataframe cells?</a></p>
  
  <p><a href="https://stackabuse.com/using-regex-for-text-manipulation-in-python/" rel="nofollow noreferrer">https://stackabuse.com/using-regex-for-text-manipulation-in-python/</a></p>