title: How to fix "cannot serialize '_io.BufferedReader' object" while multiprocessing?
body: <p>I get the following error when trying to parse a large number of web pages from a website : "Reason: 'TypeError("cannot serialize '_io.BufferedReader' object",)'. How can I fix it ?</p>
  
  <p>full error message is :</p>
  
  <pre><code>File "main.py", line 29, in &lt;module&gt;
      records = p.map(defs.scrape,state_urls)
    File "C:\Users\Utilisateur\Anaconda3\lib\multiprocessing\pool.py", line 266, in map
      return self._map_async(func, iterable, mapstar, chunksize).get()
    File "C:\Users\Utilisateur\Anaconda3\lib\multiprocessing\pool.py", line 644, in get
      raise self._value
  multiprocessing.pool.MaybeEncodingError: Error sending result: '&lt;multiprocessing.pool.ExceptionWithTraceback object at 0x0000018DD1C3D828&gt;'. Reason: 'TypeError("cannot serialize '_io.BufferedReader' object",)'
  </code></pre>
  
  <p>I browsed through some of the answers for similar questions here, namely this one (<a href="https://stackoverflow.com/questions/48761983/multiprocessing-pool-maybeencodingerror-error-sending-result-reason-typeerro">multiprocessing.pool.MaybeEncodingError: Error sending result: Reason: &#39;TypeError(&quot;cannot serialize &#39;_io.BufferedReader&#39; object&quot;,)&#39;</a>) but I don't think I'm running into the same issue, as I don't handle files directly in the scrape function. </p>
  
  <p>I tried modifying the scrape function so it returned a string and not a list (don't know why I did that) but I didn't work.</p>
  
  <p>From the main.py file :</p>
  
  <pre class="lang-py prettyprint-override"><code>from urllib.request import urlopen as uReq
  from bs4 import BeautifulSoup as soup 
  from multiprocessing import Pool
  import codecs
  import defs
  if __name__ == '__main__':
  
      filename = "some_courts_test.csv"
          # not the actual values
      courts = ["blabla", "blablabla", "blablabla","blabla"]
  
      client = defs.init_client()
      i = 1
  
      # scrapes the data from the website and puts it into a csv file
      for court in courts:
          records = []
          records_string =""
          print("creating a file for the court of : "+court)
          f = defs.init_court_file(court)
          print("generating urls for the court of "+court)        
          state_urls = defs.generate_state_urls(court)
          for url in state_urls:
              print(url)
          print("scraping creditors from : "+court)
          p = Pool(10)
  
          records = p.map(defs.scrape,state_urls)
          records_string = ''.join(records[1])
          p.terminate()
          p.join()
          for r in records_string:
              f.write(r)
          records = []
  
          f.close()
  </code></pre>
  
  <p>from the defs file:</p>
  
  <pre class="lang-py prettyprint-override"><code>def scrape(url):
          data = []
          row_string = ' '
          final_data = []
          final_string = ' '
          uClient = uReq(url)
          page_html = uClient.read()
          uClient.close()
  
          page_soup = soup(page_html, "html.parser")
          table = page_soup.find("table", {"class":"table table-striped"})
          table_body = table.find('tbody')
          rows = table_body.find_all('tr')
          for row in rows:
              cols = row.find_all('td')
              cols = [ele.text.replace(',',' ') for ele in cols] #cleans it up
  
              for ele in cols:
                  if ele:
                      data.append(ele)
                  data.append(',')
              data.append('\n')
          return(data)
  </code></pre>