class_docstr: Parallelformers end-point function
  
  Args:
      model (nn.Module): Huggingface pre-trained transformer model.
      fp16: (bool): whether use FP16 or not.
      num_gpus (int): number of GPU for parallelization.
      master_addr (str): master process address for process communication (default='127.0.0.1')
      master_port (int): master process port for process communication (default=29500)
      backend (str): distributed backend (default='nccl')
      verbose (str): logging current gpu states (one of ['detail', 'simple', None]
      init_method (str): multiprocess initialization method. (It is safe to set `init_method` to `spawn`.)
      daemon (bool): whether make process daemon or not (default=True)
  
  Notes:
      We want to use this object as a simple function rather than a class.
      So we broke the PEP8 rules and set class names that start with a lowercase letter.
  
  Examples:
      >>> # 1. Import Huggingface and Parallelformers modules
      >>> from transformers import AutoModelForCausalLM, AutoTokenizer
      >>> from parallelformers import parallelize
  
      >>> # 2. Create Huggingface model and tokenizer.
      >>> model = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-neo-2.7B")
      >>> tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-2.7B")
  
      >>> # 3. Parallelize using Parallelformers.
      >>> parallelize(model, num_gpus=4, fp16=True)
  
      >>> # 4. Do inference as usual.
      >>> inputs = tokenizer("Parallelformers is", return_tensors="pt")
      >>> outputs = model.generate(**inputs, num_beams=5, no_repeat_ngram_size=4)
      >>> print(f"Output: {tokenizer.batch_decode(outputs)[0]}")
      'Output: Parallelformers is an open-source library for parallel programming ...'
docstr: Remove all methods registered in the model
  and join all GPU processes to main process.
docstr_tok: ["Remove","all","methods","registered","in","the","model","and","join","all","GPU","processes","to","main","process","."]
code: def deparallelize(self) -> None:
      """
      Remove all methods registered in the model
      and join all GPU processes to main process.
      """
  
      if hasattr(self, "orig_methods"):
          for k, v in self.orig_methods.items():
              setattr(self.model, k, v)
  
      if hasattr(self, "_memory_logger"):
          for method in self._memory_logger:
              setattr(self.model, method, None)
  
      if hasattr(self, "processes"):
          for process in self.processes:
              process.join()
  
      torch.cuda.empty_cache()
  
imports: import os
  import traceback
  from contextlib import suppress
  from dataclasses import _is_dataclass_instance
  from typing import Any, Dict
  import torch
  import torch.multiprocessing
  from dacite import Config, from_dict
  from torch import nn
  from transformers.file_utils import ModelOutput
  from parallelformers.parallel.process import ParallelProcess
  from parallelformers.utils import rgetattr, rsetattr
label_desc: Use image node_id 6 for calling the parallelize obj's underlying member method code with example usage: obj.deparallelize() without return types