title: How to get rid of \ufeff in parsed html page
body: <p>The code is </p>
  
  <pre><code>!wget -q -O 'boroughs.html' "https://en.wikipedia.org/wiki/List_of_London_boroughs"
  
  with open('boroughs.html', encoding='utf-8-sig') as fp:
      soup = BeautifulSoup(fp,"lxml")
  
  
  data = []
  table = soup.find("table", { "class" : "wikitable sortable" })
  table_body = table.find('tbody')
  rows = table_body.find_all('tr')
  for row in rows:
      cols = row.find_all('td')
      cols = [col.text.strip() for col in cols]
      data.append([col for col in cols]) # Get rid of empty values
  data
  </code></pre>
  
  <p>I've added <code>encoding='utf-8-sig'</code> to open after some research. But in the output <a href="https://i.stack.imgur.com/LIbjG.png" rel="nofollow noreferrer">I still see  the characters \ufeff:</a> </p>
  
  <p>What puzzles me, I've even tried the hacky way with </p>
  
  <pre><code>df = df.replace(u'\ufeff', '') 
  </code></pre>
  
  <p>after adding data to pandas dataframe</p>
  
  <p>And the characters are still there. </p>