class_docstr: Memory Map Index and Binary file for RETRO DATA.
  It provides `chunks` to the original MMap data so data can be fetched at both document and chunk level.
  It can be used both for training data and Retrieval Data.
  Retrieval Dataset adds an extra `chunk_size` padded tokens at the end of each document. '
  `self._index.retrieval_db` is indicating whether it is retrieval dataset or not.
  
  It is built by `preprocess_data_for_megatron.py` script.
class_docstr_tok: ["Memory","Map","Index","and","Binary","file","for","RETRO","DATA",".","It","provides","`","chunks","`","to","the","original","MMap","data","so","data","can","be","fetched","at","both","document","and","chunk","level",".","It","can","be","used","both","for","training","data","and","Retrieval","Data",".","Retrieval","Dataset","adds","an","extra","`","chunk_size","`","padded","tokens","at","the","end","of","each","document",".","'","`","self._index.retrieval_db","`","is","indicating","whether","it","is","retrieval","dataset","or","not",".","It","is","built","by","`","preprocess_data_for_megatron.py","`","script","."]
docstr: get the chunk id from document idx and offset position.
          
docstr_tok: ["get","the","chunk","id","from","document","idx","and","offset","position","."]
code: def get_chunk_id(self, idx, offset=0):
      """get the chunk id from document idx and offset position."""
      # make sure offset is a multiple of chunk_size
      assert offset % self._index.chunk_size == 0
      return self._index.get_chunk_id(idx, offset)
  
imports: import os
  import shutil
  import struct
  from functools import lru_cache
  from itertools import accumulate
  from typing import List
  import numpy
  import torch
  from nemo.utils import logging
label_desc: Use image node_id 8 for calling the MMapRetrievalIndexedDataset obj's underlying member method code with example usage: obj.get_chunk_id(idx, offset) and returns: self