class_docstr: Persistent data structure for replay buffer
  
  Features:
  - Perfectly compatible with collections.RandomAccessQueue
  - Persist replay buffer data on storage to survive restart
    - [todo] remove popleft'd data from disk
  - Reuse replay buffer data to another training session
    - Track back the replay buffer lineage
  Non-it-is-for:
  - Extend replay buffer by spilling to the disk
  
  TODOs
  - Optimize writes; buffered writes with threads or something
  
  Arguments:
      basedir (str): Path to the directory where replay buffer data is stored.
      maxlen (int): Max length of queue. Appended data beyond
          this limit is only removed from memory, not from storage.
      ancestor (str): Path to pre-generated replay buffer.
      logger: logger
class_docstr_tok: ["Persistent","data","structure","for","replay","buffer","Features",":","-","Perfectly","compatible","with","collections.RandomAccessQueue","-","Persist","replay","buffer","data","on","storage","to","survive","restart","-","[","todo","]","remove","popleft","'d","data","from","disk","-","Reuse","replay","buffer","data","to","another","training","session","-","Track","back","the","replay","buffer","lineage","Non-it-is-for",":","-","Extend","replay","buffer","by","spilling","to","the","disk","TODOs","-","Optimize","writes",";","buffered","writes","with","threads","or","something","Arguments",":","basedir","(","str",")",":","Path","to","the","directory","where","replay","buffer","data","is","stored",".","maxlen","(","int",")",":","Max","length","of","queue",".","Appended","data","beyond","this","limit","is","only","removed","from","memory",",","not","from","storage",".","ancestor","(","str",")",":","Path","to","pre-generated","replay","buffer",".","logger",":","logger"]
docstr: Simple implementation
code: def _load_ancestor(self, ancestor, num_data_needed):
      """Simple implementation"""
      ancestor_metafile = PersistentRandomAccessQueue._meta_file_name(
          ancestor
      )
      with self.fs.open(ancestor_metafile, "rb") as fp:
          meta = pickle.load(fp)
          assert isinstance(meta, dict)
      if self.logger:
          self.logger.info("Loading buffer data from %s", ancestor)
  
      datadirs = []
      saved_comm_size = meta["comm_size"]
      n_data_dirs = (
          saved_comm_size + self.comm_size - 1
      ) // self.comm_size
      data_dir_i = self.comm_rank
      for _ in range(n_data_dirs):
          data_dir_i = data_dir_i % saved_comm_size
          datadirs.append(
              os.path.join(ancestor, "rank{}".format(data_dir_i))
          )
          data_dir_i += self.comm_size
  
      length = 0
      for datadir in datadirs:
          reader = _ChunkReader(datadir, self.fs)
          gen = 0
          while True:
              filename = os.path.join(
                  datadir, _INDEX_FILENAME_FORMAT.format(gen)
              )
              if not self.fs.exists(filename):
                  break
              if self.logger:
                  self.logger.debug(
                      "read_chunk_index from %s, gen=%d", datadir, gen
                  )
              length += len(list(reader.read_chunk_index(gen)))
              gen += 1
  
      if length < num_data_needed and meta["ancestor"] is not None:
          self._load_ancestor(
              meta["ancestor"], num_data_needed - length
          )
  
      for datadir in datadirs:
          reader = _ChunkReader(datadir, self.fs)
          rank_data = []
          maxlen = num_data_needed - len(self.buffer)
          if maxlen <= 0:
              break
          _ = reader.read_chunks(maxlen, rank_data)
          if self.logger:
              self.logger.info(
                  "%d data loaded to buffer (rank=%d)",
                  len(rank_data),
                  self.comm_rank,
              )
          self.buffer.extend(rank_data)
      return meta
  
imports: import binascii
  import collections
  import os
  import pickle
  from datetime import datetime
  from struct import calcsize, pack, unpack
  from pfrl.collections.random_access_queue import RandomAccessQueue
label_desc: Use image node_id 21 for calling the PersistentRandomAccessQueue obj's underlying member method code with example usage: obj._load_ancestor(ancestor, num_data_needed) and returns: meta