class_docstr: A [Map style dataset](https://pytorch.org/docs/stable/data.html)
  for iterating over the wikitext dataset. Note that this assumes
  the dataset can fit in memory. For larger datasets
  you'd want to shard them and use an iterable dataset (eg: see
  [Infinibatch](https://github.com/microsoft/infinibatch))
  
  Args:
      Dataset (datasets.arrow_dataset.Dataset):
          The wikitext dataset
      masking_function (Callable[[str], Tuple[List[int], List[int]]])
          The masking function. To generate one training instance,
          the masking function is applied to the `text` of a dataset
          record
class_docstr_tok: ["A","[","Map","style","dataset","]","(","https",":","\/\/pytorch.org\/docs\/stable\/data.html",")","for","iterating","over","the","wikitext","dataset",".","Note","that","this","assumes","the","dataset","can","fit","in","memory",".","For","larger","datasets","you","'d","want","to","shard","them","and","use","an","iterable","dataset","(","eg",":","see","[","Infinibatch","]","(","https",":","\/\/github.com\/microsoft\/infinibatch",")",")","Args",":","Dataset","(","datasets.arrow_dataset.Dataset",")",":","The","wikitext","dataset","masking_function","(","Callable","[","[","str","]",",","Tuple","[","List","[","int","]",",","List","[","int","]","]","]",")","The","masking","function",".","To","generate","one","training","instance",",","the","masking","function","is","applied","to","the","`","text","`","of","a","dataset","record"]
code: def __len__(self) -> int:
      return len(self.dataset)
  
imports: import datetime
  import json
  import pathlib
  import re
  import string
  from functools import partial
  from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, TypeVar, Union
  import random
  import datasets
  import fire
  import loguru
  import numpy
  import pytz
  import sh
  import torch
  import torch.nn
  from torch.utils.data import DataLoader, Dataset
  from torch.utils.tensorboard import SummaryWriter
  from transformers import AutoTokenizer, PreTrainedTokenizer, PreTrainedTokenizerFast
  from transformers.models.roberta import RobertaConfig, RobertaModel
  from transformers.models.roberta.modeling_roberta import RobertaLMHead, RobertaPreTrainedModel
  from deepspeed.accelerator import get_accelerator
label_desc: Use image node_id 2 for calling the WikiTextMLMDataset obj's underlying member method code with example usage: obj.__len__() and returns: len