class_docstr: Cross-lingual BERT dataset similar to the translation-language modeling objective in the XLM paper (https://arxiv.org/abs/1901.07291)
code: def __init__(
      self,
      src_dataset_prefix: str,
      tgt_dataset_prefix: str,
      src_tokenizer: TokenizerSpec,
      tgt_tokenizer: TokenizerSpec,
      max_src_seq_length: int,
      max_tgt_seq_length: int,
      seed: int = 1234,
      max_num_samples: int = None,
      masked_lm_prob: float = 0.15,
  ):
      super().__init__(
          src_dataset_prefix=src_dataset_prefix,
          tgt_dataset_prefix=tgt_dataset_prefix,
          src_tokenizer=src_tokenizer,
          tgt_tokenizer=tgt_tokenizer,
          max_src_seq_length=max_src_seq_length,
          max_tgt_seq_length=max_tgt_seq_length,
          seed=seed,
          max_num_samples=max_num_samples,
      )
      assert src_tokenizer == tgt_tokenizer
      # Vocab stuff.
      self.vocab = src_tokenizer.vocab
      self.vocab_id_list = list(
          range(self.src_tokenizer.original_vocab_size)
      )
      self.vocab_id_to_token_dict = {
          idx: token for idx, token in enumerate(self.vocab)
      }
      self.cls_id = src_tokenizer.cls_id
      self.sep_id = src_tokenizer.sep_id
      self.mask_id = src_tokenizer.mask_id
      self.pad_id = src_tokenizer.pad_id
      self.bos_id = src_tokenizer.bos_id
      self.eos_id = src_tokenizer.eos_id
      self.max_seq_length = max_src_seq_length + max_tgt_seq_length
      self.masked_lm_prob = masked_lm_prob
  
imports: from typing import Dict, List
  import numpy
  from nemo.collections.common.tokenizers.tokenizer_spec import TokenizerSpec
  from nemo.collections.nlp.data.common.sequence_to_sequence_dataset import BinarizedMemmapSequenceToSequenceDataset, TextMemmapSequenceToSequenceDataset
  from nemo.collections.nlp.data.language_modeling.megatron.bert_dataset import build_training_sample
  from nemo.collections.nlp.data.language_modeling.megatron.length_distribution_type import LengthDistribution
  from nemo.collections.nlp.data.language_modeling.megatron.t5_dataset import T5Dataset
  from nemo.collections.nlp.data.language_modeling.megatron.ul2_dataset import UL2Dataset
label_desc: Use image node_id 1 to create a new CrossLingualBERTDataset object from inherited base classes: BinarizedMemmapSequenceToSequenceDataset with example: obj = CrossLingualBERTDataset(src_dataset_prefix, tgt_dataset_prefix, src_tokenizer, tgt_tokenizer, max_src_seq_length, max_tgt_seq_length, seed, max_num_samples, masked_lm_prob)