docstr: This tests that the model has been loaded and can be used correctly on a multi-GPU setup.
  Let's just try to load a model on 2 GPUs and see if it works. The model we test has ~2GB of total, 3GB should suffice
docstr_tok: ["This","tests","that","the","model","has","been","loaded","and","can","be","used","correctly","on","a","multi-GPU","setup",".","Let","'s","just","try","to","load","a","model","on","2","GPUs","and","see","if","it","works",".","The","model","we","test","has","~2GB","of","total",",","3GB","should","suffice"]
code: def test_multi_gpu_loading(self):
      r"""
      This tests that the model has been loaded and can be used correctly on a multi-GPU setup.
      Let's just try to load a model on 2 GPUs and see if it works. The model we test has ~2GB of total, 3GB should suffice
      """
  
      memory_mapping = {0: "1GB", 1: "2GB"}
      model_parallel = AutoModelForCausalLM.from_pretrained(
          self.model_name,
          load_in_8bit=True,
          max_memory=memory_mapping,
          device_map="auto",
      )
  
      def get_list_devices(model):
          list_devices = []
          for _, module in model.named_children():
              if len(list(module.children())) > 0:
                  list_devices.extend(get_list_devices(module))
              else:
                  # Do a try except since we can encounter Dropout modules that does not
                  # have any device set
                  try:
                      list_devices.append(
                          next(module.parameters()).device.index
                      )
                  except BaseException:
                      continue
          return list_devices
  
      list_devices = get_list_devices(model_parallel)
      # Check that we have dispatched the model into 2 separate devices
      self.assertTrue((1 in list_devices) and (0 in list_devices))
  
      # Check that inference pass works on the model
      encoded_input = self.tokenizer(
          self.input_text, return_tensors="pt"
      )
  
      # Second real batch
      output_parallel = model_parallel.generate(
          input_ids=encoded_input["input_ids"].to(0), max_new_tokens=10
      )
      self.assertEqual(
          self.tokenizer.decode(
              output_parallel[0], skip_special_tokens=True
          ),
          self.EXPECTED_OUTPUT,
      )
  
imports: import gc
  import unittest
  from transformers import AutoModel, AutoModelForCausalLM, AutoModelForSeq2SeqLM, AutoModelForSequenceClassification, AutoTokenizer, pipeline
  from transformers.testing_utils import is_torch_available, require_accelerate, require_bitsandbytes, require_torch, require_torch_gpu, require_torch_multi_gpu, slow
label_desc: Use image node_id 2 for calling the MixedInt8TestMultiGpu obj's underlying member method code with example usage: obj.test_multi_gpu_loading() and returns: list_devices