class_docstr: The goal here is to compare the embeddings generated by the model trained
  using Megatron-LM with the one from the transformers library, with a small GPT2-like model
  to ensure that the conversion from Megatron-LM to transformers has been done successfully.
  The script compares the logits of the embedding layer and the transformer layers.
  
  WARNING: It is expected that these logits will not have exactly the same statistics when running
  the code on CPU or GPU. For more info, please visit:
    - https://github.com/pytorch/pytorch/issues/76052#issuecomment-1103193548
    - https://discuss.pytorch.org/t/reproducibility-issue-between-intel-and-amd-cpus/144779/9
  
  
  You need to install tokenizers following this readme:
      - https://huggingface.co/bigscience-catalogue-data-dev/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles
  
  Tokenizer used during training:
      - https://huggingface.co/bigscience-catalogue-data-dev/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles
  
  # TODO change the script (or just add skip) when building the env with tokenizers 0.12.0
code: def test_logits(self):
      cuda_available = torch.cuda.is_available()
      model = BloomForCausalLM.from_pretrained(
          self.path_bigscience_model,
          use_cache=False,
          torch_dtype="auto",
      ).to(
          torch_device
      )  # load in bf16
      model.eval()
  
      EXAMPLE_IDS = [3478, 368, 109586, 35433, 2, 77, 132619, 3478, 368, 109586, 35433, 2, 2175, 23714, 73173, 144252, 2, 77, 132619, 3478]  # fmt: skip
  
      MEAN_LOGITS_GPU_1 = -1.823902130126953e-05
      MEAN_LOGITS_GPU_2 = 1.9431114196777344e-05
  
      tensor_ids = torch.LongTensor([EXAMPLE_IDS]).to(torch_device)
      with torch.no_grad():
          output = model(tensor_ids).logits
  
      output_gpu_1, output_gpu_2 = output.split(125440, dim=-1)
      if cuda_available:
          self.assertAlmostEqual(
              output_gpu_1.mean().item(), MEAN_LOGITS_GPU_1, places=6
          )
          self.assertAlmostEqual(
              output_gpu_2.mean().item(), MEAN_LOGITS_GPU_2, places=6
          )
      else:
          self.assertAlmostEqual(
              output_gpu_1.mean().item(), MEAN_LOGITS_GPU_1, places=6
          )  # 1e-06 precision!!
          self.assertAlmostEqual(
              output_gpu_2.mean().item(), MEAN_LOGITS_GPU_2, places=6
          )
  
imports: import math
  import unittest
  from transformers import BloomConfig, is_torch_available
  from transformers.testing_utils import require_torch, require_torch_accelerator, slow, torch_device
  from ...generation.test_utils import GenerationTesterMixin
  from ...test_configuration_common import ConfigTester
  from ...test_modeling_common import ModelTesterMixin, ids_tensor, random_attention_mask
  from ...test_pipeline_mixin import PipelineTesterMixin
label_desc: Use image node_id 4 for calling the BloomEmbeddingTest obj's underlying member method code with example usage: obj.test_logits() without return types