title: Got stuck while reading files
body: <p><strong>what Code DO's</strong></p>
  
  <p>I am trying to read each file from the folder which i have given ,And extracting some line using bs4 Soup package in python.</p>
  
  <p>I got an error reading the file that some unicode chars not able to read.</p>
  
  <p><strong>error</strong></p>
  
  <blockquote>
    <p>Traceback (most recent call last):   File "C:-----\check.py", line 25, in 
        soup=BeautifulSoup(text.read(), 'html.parser')   File "C:\Python\Python37\lib\encodings\cp1252.py",
    line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position
    3565: character maps to </p>
  </blockquote>
  
  <pre><code>   from bs4 import BeautifulSoup
     from termcolor import colored
     import re, os
  
     import requests
     path = "./brian-work/"
     freddys_library = os.listdir(path)
  def getfiles():
    for r, d, f in os.walk(path):
      for file in f:
          if '.html' in file:
              files.append(os.path.join(r, file))
    return files
  
  
  for book in getfiles():
      print("file is printed")
      print(book)
      text = open(book, "r")
      soup=BeautifulSoup(text.read(), 'html.parser')
      h1 = soup.select('h1')[0].text.strip()
      print(h1)
      if soup.find('h1'):
        h1 = soup.select('h1')[0].text.strip()
      else:
  
        print ("no h1")
        continue
  
      filename1=book.split("/")[-1]
      filename1=filename1.split(".")[0]
      print(h1.split(' ', 1)[0])
      print(filename1) 
      if h1.split(' ', 1)[0].lower() == filename1.split('-',1)[0] :
        print('+++++++++++++++++++++++++++++++++++++++++++++');
        print('same\n');
      else:
        print('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');
        print('not')
        count=count+1
  </code></pre>
  
  <p>Please help what should i correct here.</p>
  
  <p>Thanks</p>