class_docstr: Contiguous memory buffer.
  Allocate a contiguous memory of type `dtype` and size `numel`. It is
  used to reduce memory fragmentation.
  
  Usage: After the allocation, the `_start` index is set tot the first
         index of the memory. A memory chunk starting from `_start` index
         can be `allocated` for an input tensor, with the elements of the
         tensor being coppied. The buffer can be reused by resetting the
         `_start` index.
class_docstr_tok: ["Contiguous","memory","buffer",".","Allocate","a","contiguous","memory","of","type","`","dtype","`","and","size","`","numel","`",".","It","is","used","to","reduce","memory","fragmentation",".","Usage",":","After","the","allocation",",","the","`","_start","`","index","is","set","tot","the","first","index","of","the","memory",".","A","memory","chunk","starting","from","`","_start","`","index","can","be","`","allocated","`","for","an","input","tensor",",","with","the","elements","of","the","tensor","being","coppied",".","The","buffer","can","be","reused","by","resetting","the","`","_start","`","index","."]
docstr: Allocate a chunk of memory from the buffer to tensor and copy
  the values.
docstr_tok: ["Allocate","a","chunk","of","memory","from","the","buffer","to","tensor","and","copy","the","values","."]
code: def add(self, tensor):
      """Allocate a chunk of memory from the buffer to tensor and copy
      the values."""
      assert (
          tensor.dtype == self.dtype
      ), "Input tensor type {} different from buffer type {}".format(
          tensor.dtype, self.dtype
      )
      # Number of elements of the input tensor.
      tensor_numel = torch.numel(tensor)
      new_start = self._start + tensor_numel
      assert (
          new_start <= self.numel
      ), "Not enough memory left in the buffer ({} > {})".format(
          tensor_numel, self.numel - self._start
      )
      # New tensor is a view into the memory.
      new_tensor = self.data[self._start : new_start]
      self._start = new_start
      new_tensor = new_tensor.view(tensor.shape)
      new_tensor.copy_(tensor)
      # Return a pointer to the new tensor.
      return new_tensor
  
file_path: megatron-lm/megatron/memory.py
imports: import torch
label_desc: Use image node_id 5 for calling the MemoryBuffer obj's underlying member method code with example usage: obj.add(tensor) and returns: new_tensor