title: Preprocessing text data on many columns from a data frame using python
body: <p>I'm looking for an answer <a href="https://stackoverflow.com/questions/48032578/preprocessing-text-analysis-on-many-columns-from-a-dataframe">like</a> this but in python. How can I do text preprocessing on multiple columns? I have two text columns see <a href="https://i.stack.imgur.com/5WvUp.png" rel="nofollow noreferrer">screenshots</a>. To do the cleaning work, I have to do twice to each column (see my code). Is there any clever way to do a similar task? Thanks!</p>
  
  <pre><code>import requests
  
  from bs4 import BeautifulSoup #html.parser'
  df['Summary'] = [BeautifulSoup(text).get_text() for text in df['Summary']]
  df['Text'] = [BeautifulSoup(text).get_text() for text in df['Text']]
  
  
  df.loc[:,"Text"] = df.Text.apply(lambda x : str.lower(x))
  df.loc[:,"Summary"] = df.Summary.apply(lambda x : str.lower(x))   
  
  #remove punctuation.
  df["Text"] = df['Text'].str.replace('[^\w\s]','')
  df["Summary"] = df['Summary'].str.replace('[^\w\s]','')
  </code></pre>