class_docstr: The goal here is to compare the embeddings generated by the model trained
  using Megatron-LM with the one from the transformers library, with a small GPT2-like model
  to ensure that the conversion from Megatron-LM to transformers has been done successfully.
  The script compares the logits of the embedding layer and the transformer layers.
  
  WARNING: It is expected that these logits will not have exactly the same statistics when running
  the code on CPU or GPU. For more info, please visit:
    - https://github.com/pytorch/pytorch/issues/76052#issuecomment-1103193548
    - https://discuss.pytorch.org/t/reproducibility-issue-between-intel-and-amd-cpus/144779/9
  
  
  You need to install tokenizers following this readme:
      - https://huggingface.co/bigscience-catalogue-data-dev/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles
  
  Tokenizer used during training:
      - https://huggingface.co/bigscience-catalogue-data-dev/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles
  
  # TODO change the script (or just add skip) when building the env with tokenizers 0.12.0
class_docstr_tok: ["The","goal","here","is","to","compare","the","embeddings","generated","by","the","model","trained","using","Megatron-LM","with","the","one","from","the","transformers","library",",","with","a","small","GPT2-like","model","to","ensure","that","the","conversion","from","Megatron-LM","to","transformers","has","been","done","successfully",".","The","script","compares","the","logits","of","the","embedding","layer","and","the","transformer","layers",".","WARNING",":","It","is","expected","that","these","logits","will","not","have","exactly","the","same","statistics","when","running","the","code","on","CPU","or","GPU",".","For","more","info",",","please","visit",":","-","https",":","\/\/github.com\/pytorch\/pytorch\/issues\/76052","#","issuecomment-1103193548","-","https",":","\/\/discuss.pytorch.org\/t\/reproducibility-issue-between-intel-and-amd-cpus\/144779\/9","You","need","to","install","tokenizers","following","this","readme",":","-","https",":","\/\/huggingface.co\/bigscience-catalogue-data-dev\/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles","Tokenizer","used","during","training",":","-","https",":","\/\/huggingface.co\/bigscience-catalogue-data-dev\/byte-level-bpe-tokenizer-no-norm-250k-whitespace-and-eos-regex-alpha-v3-dedup-lines-articles","#","TODO","change","the","script","(","or","just","add","skip",")","when","building","the","env","with","tokenizers","0.12.0"]
imports: import math
  import unittest
  from transformers import BloomConfig, is_torch_available
  from transformers.testing_utils import require_torch, require_torch_gpu, slow, torch_device
  from ...generation.test_generation_utils import GenerationTesterMixin
  from ...test_configuration_common import ConfigTester
  from ...test_modeling_common import ModelTesterMixin, ids_tensor, random_attention_mask
label_desc: Use image node_id 2 for calling the BloomEmbeddingTest obj's underlying member method code with example usage: obj.test_embeddings() without return types