class_docstr: A distributed retriever built on top of the ``Ray`` API, a library
  for building distributed applications (https://docs.ray.io/en/master/).
  package. During training, all training workers initialize their own
  instance of a `RagRayDistributedRetriever`, and each instance of
  this distributed retriever shares a common set of Retrieval Ray
  Actors (https://docs.ray.io/en/master/walkthrough.html#remote
  -classes-actors) that load the index on separate processes. Ray
  handles the communication between the `RagRayDistributedRetriever`
  instances and the remote Ray actors. If training is done in a
  non-distributed setup, the index will simply be loaded in the same
  process as the training worker and Ray will not be used.
  
  Args:
      config (:class:`~transformers.RagConfig`):
          The configuration of the RAG model this Retriever is used with. Contains parameters indicating which ``Index`` to build.
      question_encoder_tokenizer (:class:`~transformers.PreTrainedTokenizer`):
          The tokenizer that was used to tokenize the question.
          It is used to decode the question and then use the generator_tokenizer.
      generator_tokenizer (:class:`~transformers.PreTrainedTokenizer`):
          The tokenizer used for the generator part of the RagModel.
      retrieval_workers (:obj:`List[ray.ActorClass(RayRetriever)]`): A list of already initialized `RayRetriever` actors.
          These actor classes run on remote processes and are responsible for performing the index lookup.
      index (:class:`~transformers.retrieval_rag.Index`, optional, defaults to the one defined by the configuration):
          If specified, use this index instead of the one built using the configuration
class_docstr_tok: ["A","distributed","retriever","built","on","top","of","the","``","Ray","``","API",",","a","library","for","building","distributed","applications","(","https",":","\/\/docs.ray.io\/en\/master\/",")",".","package",".","During","training",",","all","training","workers","initialize","their","own","instance","of","a","`","RagRayDistributedRetriever","`",",","and","each","instance","of","this","distributed","retriever","shares","a","common","set","of","Retrieval","Ray","Actors","(","https",":","\/\/docs.ray.io\/en\/master\/walkthrough.html","#","remote","-classes-actors",")","that","load","the","index","on","separate","processes",".","Ray","handles","the","communication","between","the","`","RagRayDistributedRetriever","`","instances","and","the","remote","Ray","actors",".","If","training","is","done","in","a","non-distributed","setup",",","the","index","will","simply","be","loaded","in","the","same","process","as","the","training","worker","and","Ray","will","not","be","used",".","Args",":","config","(",":","class",":","`","~transformers.RagConfig","`",")",":","The","configuration","of","the","RAG","model","this","Retriever","is","used","with",".","Contains","parameters","indicating","which","``","Index","``","to","build",".","question_encoder_tokenizer","(",":","class",":","`","~transformers.PreTrainedTokenizer","`",")",":","The","tokenizer","that","was","used","to","tokenize","the","question",".","It","is","used","to","decode","the","question","and","then","use","the","generator_tokenizer",".","generator_tokenizer","(",":","class",":","`","~transformers.PreTrainedTokenizer","`",")",":","The","tokenizer","used","for","the","generator","part","of","the","RagModel",".","retrieval_workers","(",":","obj",":","`","List","[","ray.ActorClass","(","RayRetriever",")","]","`",")",":","A","list","of","already","initialized","`","RayRetriever","`","actors",".","These","actor","classes","run","on","remote","processes","and","are","responsible","for","performing","the","index","lookup",".","index","(",":","class",":","`","~transformers.retrieval_rag.Index","`",",","optional",",","defaults","to","the","one","defined","by","the","configuration",")",":","If","specified",",","use","this","index","instead","of","the","one","built","using","the","configuration"]
code: def get_tokenizers(
      cls, retriever_name_or_path, indexed_dataset=None, **kwargs
  ):
      return super(RagRayDistributedRetriever, cls).get_tokenizers(
          retriever_name_or_path, indexed_dataset, **kwargs
      )
  
file_path: transformers/examples/research_projects/rag/distributed_ray_retriever.py
imports: import logging
  import random
  import ray
  from transformers import RagConfig, RagRetriever, RagTokenizer
  from transformers.models.rag.retrieval_rag import CustomHFIndex
label_desc: Use image node_id 4 for calling the RagRayDistributedRetriever obj's underlying member method code with example usage: obj.get_tokenizers(cls, retriever_name_or_path, indexed_dataset) and returns: super