title: Python Unicode Not Parsing Apostrophe's
body: <p>I'm trying to parse a title of the following NYT article: <a href="https://www.nytimes.com/2018/01/14/us/politics/david-perdue-trump-shithole.html" rel="nofollow noreferrer">https://www.nytimes.com/2018/01/14/us/politics/david-perdue-trump-shithole.html</a></p>
  
  <p>The title I would like my code to parse is "Hopes Dim for DACA Deal as Lawmakers Battle Over Trump’s Immigration Remarks - The New York Times." I get this result when I run print soup.html.head.title when running with the debugger below. Other than capturing the stdout in a variable (which seems roundabout), is there a smarter way I can get the text I want? </p>
  
  <p><em>Alternate Try #1</em> </p>
  
  <pre><code>(Pdb) str(soup.html.head.title)
  '&lt;title&gt;Hopes Dim for DACA Deal as Lawmakers Battle Over Trump\xe2\x80\x99s Immigration Remarks - The New York Times&lt;/title&gt;'
  </code></pre>
  
  <p><em>Alternate Try #2</em> </p>
  
  <pre><code>(Pdb) soup.html.head.title.encode('utf-8')
  '&lt;title&gt;Hopes Dim for DACA Deal as Lawmakers Battle Over Trump\xe2\x80\x99s Immigration Remarks - The New York Times&lt;/title&gt;'
  </code></pre>
  
  <p><em>Alternate Try #3</em></p>
  
  <pre><code>(Pdb) soup.html.head.title.encode('ascii')
  '&lt;title&gt;Hopes Dim for DACA Deal as Lawmakers Battle Over Trump&amp;#8217;s Immigration Remarks - The New York Times&lt;/title&gt;'
  </code></pre>
  
  <p>Code:</p>
  
  <pre><code>from __future__ import division
  
  import regex as re
  import string
  import urllib2
  import pdb
  from collections import Counter
  
  from bs4 import BeautifulSoup
  from cookielib import CookieJar
  
  PARSER_TYPE = 'html.parser'
  
  class NYT(object):
      def __init__(self, url, title='test-title'):
          self.url = url
          self.title = get_title(url)
  
  def get_title(url):
      cj = CookieJar()
      opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
      p = opener.open(url)
      soup = BeautifulSoup(p.read(), PARSER_TYPE)
      title = soup.html.head.title.string
      pdb.set_trace() # trying a few different things here
      title = re.sub(r'[^\x00-\x7F]+',"", title).replace(" - The New York Times", "")
      return title
  </code></pre>