class_docstr: Wikitext103 dataset token-level.
code: def generate_samples(self, data_dir, tmp_dir, dataset_split):
      train_file, valid_file, test_file = _maybe_download_corpus(
          tmp_dir, self.vocab_type
      )
  
      if dataset_split == problem.DatasetSplit.TRAIN:
          filepath = train_file
          if self.vocab_type == text_problems.VocabType.TOKEN:
              _build_vocab(train_file, data_dir, self.vocab_filename)
  
      elif dataset_split == problem.DatasetSplit.EVAL:
          filepath = valid_file
  
      elif dataset_split == problem.DatasetSplit.TEST:
          filepath = test_file
  
      def _generate_samples():
          with tf.gfile.GFile(filepath, "r") as f:
              for line in f:
                  line = " ".join(line.strip().split())
                  if line:
                      yield {"targets": line}
  
      return _generate_samples()
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import os
  import zipfile
  from tensor2tensor.data_generators import generator_utils
  from tensor2tensor.data_generators import problem
  from tensor2tensor.data_generators import text_encoder
  from tensor2tensor.data_generators import text_problems
  from tensor2tensor.utils import registry
  import tensorflow.compat.v1
label_desc: Use image node_id 4 for calling the LanguagemodelWikitext103 obj's underlying member method code with example usage: obj.generate_samples(data_dir, tmp_dir, dataset_split) and returns: _generate_samples