title: python compressed 4Gb bz2 EOFError: end of stream was already found nested subfolders
body: <p>I'm trying to read a specific file from a compressed file bz2 using python.</p>
  
  <pre><code>tar = tarfile.open(filename, "r|bz2", bufsize=57860311)
  
  for tarinfo in tar:
      print tarinfo.name, "is", tarinfo.size, "bytes in size and is",
      if tarinfo.isreg():
          print "a regular file."
          # read the file
          f = tar.extractfile(tarinfo)
          #print f.read()
      elif tarinfo.isdir():
          print "a directory."
      else:
          print "something else."
  tar.close()
  </code></pre>
  
  <p>But at the end I got the error:</p>
  
  <pre><code>/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in read(self, size)
      577             buf = "".join(t)
      578         else:
  --&gt; 579             buf = self._read(size)
      580         self.pos += len(buf)
      581         return buf
  
  /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in _read(self, size)
      594                 break
      595             try:
  --&gt; 596                 buf = self.cmp.decompress(buf)
      597             except IOError:
      598                 raise ReadError("invalid compressed data")
  
  EOFError: end of stream was already found
  </code></pre>
  
  <p>I also tried to list the files within the tar through 'tar.list()' and again ...</p>
  
  <pre><code>-rwxr-xr-x lindauer/or3uunp          0 2013-05-21 00:58:36 r3.2/
  -rw-r--r-- lindauer/or3uunp       6057 2012-01-05 14:41:00 r3.2/readme.txt
  -rw-r--r-- lindauer/or3uunp      44732 2012-01-04 10:08:54 r3.2/psychometric.csv
  -rw-r--r-- lindauer/or3uunp   57860309 2012-01-04 09:58:20 r3.2/logon.csv
  
  /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in _read(self, size)
      594                 break
      595             try:
  --&gt; 596                 buf = self.cmp.decompress(buf)
      597             except IOError:
      598                 raise ReadError("invalid compressed data")
  
  EOFError: end of stream was already found
  </code></pre>
  
  <p>I listed the files inside the archive using the tar command. Here is the result:</p>
  
  <pre><code>tar -tvf r3.2.tar.bz2
  drwxr-xr-x  0 lindauer or3uunp     0 May 21  2013 r3.2/
  -rw-r--r--  0 lindauer or3uunp  6057 Jan  5  2012 r3.2/readme.txt
  -rw-r--r--  0 lindauer or3uunp 44732 Jan  4  2012 r3.2/psychometric.csv
  -rw-r--r--  0 lindauer or3uunp 57860309 Jan  4  2012 r3.2/logon.csv
  -rw-r--r--  0 lindauer or3uunp 12494829865 Jan  5  2012 r3.2/http.csv
  -rw-r--r--  0 lindauer or3uunp  1066622500 Jan  5  2012 r3.2/email.csv
  -rw-r--r--  0 lindauer or3uunp   218962503 Jan  5  2012 r3.2/file.csv
  -rw-r--r--  0 lindauer or3uunp    29156988 Jan  4  2012 r3.2/device.csv
  drwxr-xr-x  0 lindauer or3uunp           0 May 20  2013 r3.2/LDAP/
  -rw-r--r--  0 lindauer or3uunp      140956 Jan  4  2012 r3.2/LDAP/2011-01.csv
  -rw-r--r--  0 lindauer or3uunp      147370 Jan  4  2012 r3.2/LDAP/2010-05.csv
  -rw-r--r--  0 lindauer or3uunp      149221 Jan  4  2012 r3.2/LDAP/2010-02.csv
  -rw-r--r--  0 lindauer or3uunp      141717 Jan  4  2012 r3.2/LDAP/2010-12.csv
  -rw-r--r--  0 lindauer or3uunp      148931 Jan  4  2012 r3.2/LDAP/2010-03.csv
  -rw-r--r--  0 lindauer or3uunp      147370 Jan  4  2012 r3.2/LDAP/2010-04.csv
  -rw-r--r--  0 lindauer or3uunp      149793 Jan  4  2012 r3.2/LDAP/2009-12.csv
  -rw-r--r--  0 lindauer or3uunp      143979 Jan  4  2012 r3.2/LDAP/2010-09.csv
  -rw-r--r--  0 lindauer or3uunp      145591 Jan  4  2012 r3.2/LDAP/2010-07.csv
  -rw-r--r--  0 lindauer or3uunp      139444 Jan  4  2012 r3.2/LDAP/2011-03.csv
  -rw-r--r--  0 lindauer or3uunp      142347 Jan  4  2012 r3.2/LDAP/2010-11.csv
  -rw-r--r--  0 lindauer or3uunp      138285 Jan  4  2012 r3.2/LDAP/2011-04.csv
  -rw-r--r--  0 lindauer or3uunp      149793 Jan  4  2012 r3.2/LDAP/2010-01.csv
  -rw-r--r--  0 lindauer or3uunp      146008 Jan  4  2012 r3.2/LDAP/2010-06.csv
  -rw-r--r--  0 lindauer or3uunp      144711 Jan  4  2012 r3.2/LDAP/2010-08.csv
  -rw-r--r--  0 lindauer or3uunp      137967 Jan  4  2012 r3.2/LDAP/2011-05.csv
  -rw-r--r--  0 lindauer or3uunp      140085 Jan  4  2012 r3.2/LDAP/2011-02.csv
  -rw-r--r--  0 lindauer or3uunp      143420 Jan  4  2012 r3.2/LDAP/2010-10.csv
  -r--r--r--  0 lindauer or3uunp        3923 Jan  4  2012 r3.2/license.txt
  </code></pre>
  
  <p>I think this is due to the fact the archive has subfolders and for some reason python libraries have problems in dealing with subfolders extractions? </p>
  
  <p>I also tried to open the tar file manually and I have no problems so I don't think the file is corrupted. Any help appreciated.</p>