title: Python: How to Remove range of Characters \x91\x87\xf0\x9f\x91\x87 from File
body: <p>I have this file with some lines that contain some unicode literals like:
   "b'Who\xe2\x80\x99s he?\n\nA fan rushed the field to join the Cubs\xe2\x80\x99 celebration after Jake Arrieta\xe2\x80\x99s no-hitter."</p>
  
  <p>I want to remove those xe2\x80\x99 like characters.</p>
  
  <p>I can remove them if I declare a string that contains these characters but my solutions don't work when reading from a CSV file. I used pandas to read the file.</p>
  
  <p>SOLUTIONS TRIED
  1.Regex
  2.Decoding and Encoding
  3.Lambda</p>
  
  <ol>
  <li>Regex Solution</li>
  </ol>
  
  <pre><code>line =  "b'Who\xe2\x80\x99s he?\n\nA fan rushed the field to join the Cubs\xe2\x80\x99 celebration after Jake Arrieta\xe2\x80\x99s no-hitter."
  code = (re.sub(r'[^\x00-\x7f]',r'', line))
  print (code)
  </code></pre>
  
  <ol start="2">
  <li>LAMBDA SOLUTION</li>
  </ol>
  
  <pre><code>stripped = lambda s: "".join(i for i in s if 31 &lt; ord(i) &lt; 127)
  code2 = stripped(line)
  print(code2)
  </code></pre>
  
  <ol start="3">
  <li>ENCODING SOLUTION</li>
  </ol>
  
  <pre><code>code3 = (line.encode('ascii', 'ignore')).decode("utf-8")
  print(code3)
  </code></pre>
  
  <p>HOW FILE WAS READ</p>
  
  <pre><code>df = pandas.read_csv('file.csv',encoding = "utf-8")
  for index, row in df.iterrows():
      print(stripped(row['text']))
      print(re.sub(r'[^\x00-\x7f]',r'', row['text']))
      print(row['text'].encode('ascii', 'ignore')).decode("utf-8"))
  
  </code></pre>
  
  <p>SUGGESTED METHOD</p>
  
  <pre><code>df = pandas.read_csv('file.csv',encoding = "utf-8")
  
  for index, row in df.iterrows():
      en = row['text'].encode()
      print(type(en))
      newline = en.decode('utf-8')
      print(type(newline))
      print(repr(newline))
      print(newline.encode('ascii', 'ignore'))
      print(newline.encode('ascii', 'replace')) 
  </code></pre>