class_docstr: Reads the TriviaQA dataset into a ``Dataset`` containing ``Instances`` with four fields:
  ``question`` (a ``TextField``), ``passage`` (another ``TextField``), ``span_start``, and
  ``span_end`` (both ``IndexFields``).
  
  TriviaQA is split up into several JSON files defining the questions, and a lot of text files
  containing crawled web documents.  We read these from a gzipped tarball, to avoid having to
  have millions of individual files on a filesystem.
  
  Because we need to read both train and validation files from the same tarball, we take the
  tarball itself as a constructor parameter, and take the question file as the argument to
  ``read``.  This means that you should give the path to the tarball in the ``dataset_reader``
  parameters in your experiment configuration file, and something like ``"wikipedia-train.json"``
  for the ``train_data_path`` and ``validation_data_path``.
  
  Parameters
  ----------
  base_tarball_path : ``str``
      This is the path to the main ``tar.gz`` file you can download from the TriviaQA website,
      with directories ``evidence`` and ``qa``.
  unfiltered_tarball_path : ``str``, optional
      This is the path to the "unfiltered" TriviaQA data that you can download from the TriviaQA
      website, containing just question JSON files that point to evidence files in the base
      tarball.
  tokenizer : ``Tokenizer``, optional
      We'll use this tokenizer on questions and evidence passages, defaulting to
      ``WordTokenizer`` if none is provided.
  token_indexers : ``Dict[str, TokenIndexer]``, optional
      Determines how both the question and the evidence passages are represented as arrays.  See
      :class:`TokenIndexer`.  Default is to have a single word ID for every token.
class_docstr_tok: ["Reads","the","TriviaQA","dataset","into","a","``","Dataset","``","containing","``","Instances","``","with","four","fields",":","``","question","``","(","a","``","TextField","``",")",",","``","passage","``","(","another","``","TextField","``",")",",","``","span_start","``",",","and","``","span_end","``","(","both","``","IndexFields","``",")",".","TriviaQA","is","split","up","into","several","JSON","files","defining","the","questions",",","and","a","lot","of","text","files","containing","crawled","web","documents",".","We","read","these","from","a","gzipped","tarball",",","to","avoid","having","to","have","millions","of","individual","files","on","a","filesystem",".","Because","we","need","to","read","both","train","and","validation","files","from","the","same","tarball",",","we","take","the","tarball","itself","as","a","constructor","parameter",",","and","take","the","question","file","as","the","argument","to","``","read","``",".","This","means","that","you","should","give","the","path","to","the","tarball","in","the","``","dataset_reader","``","parameters","in","your","experiment","configuration","file",",","and","something","like","``","``","wikipedia-train.json","''","``","for","the","``","train_data_path","``","and","``","validation_data_path","``",".","Parameters","--","--","--","--","--","base_tarball_path",":","``","str","``","This","is","the","path","to","the","main","``","tar.gz","``","file","you","can","download","from","the","TriviaQA","website",",","with","directories","``","evidence","``","and","``","qa","``",".","unfiltered_tarball_path",":","``","str","``",",","optional","This","is","the","path","to","the","``","unfiltered","''","TriviaQA","data","that","you","can","download","from","the","TriviaQA","website",",","containing","just","question","JSON","files","that","point","to","evidence","files","in","the","base","tarball",".","tokenizer",":","``","Tokenizer","``",",","optional","We","'ll","use","this","tokenizer","on","questions","and","evidence","passages",",","defaulting","to","``","WordTokenizer","``","if","none","is","provided",".","token_indexers",":","``","Dict","[","str",",","TokenIndexer","]","``",",","optional","Determines","how","both","the","question","and","the","evidence","passages","are","represented","as","arrays",".","See",":","class",":","`","TokenIndexer","`",".","Default","is","to","have","a","single","word","ID","for","every","token","."]
code: def text_to_instance(
      self,  # type: ignore
      question_text,
      passage_text,
      token_spans=None,
      answer_texts=None,
      question_tokens=None,
      passage_tokens=None,
  ):
      # pylint: disable=arguments-differ
      if not question_tokens:
          question_tokens = self._tokenizer.tokenize(question_text)
      if not passage_tokens:
          passage_tokens = self._tokenizer.tokenize(passage_text)
      return util.make_reading_comprehension_instance(
          question_tokens,
          passage_tokens,
          self._token_indexers,
          passage_text,
          token_spans,
          answer_texts,
      )
  
file_path: magnitude/pymagnitude/third_party/allennlp/data/dataset_readers/reading_comprehension/triviaqa.py
imports: from __future__ import absolute_import
  import json
  import logging
  import os
  import tarfile
  from allennlp.common.file_utils import cached_path
  from allennlp.data.dataset_readers.dataset_reader import DatasetReader
  from allennlp.data.instance import Instance
  from allennlp.data.dataset_readers.reading_comprehension import util
  from allennlp.data.token_indexers import SingleIdTokenIndexer, TokenIndexer
  from allennlp.data.tokenizers import Token, Tokenizer, WordTokenizer
label_desc: Use image node_id 4 for calling the TriviaQaReader obj's underlying member method code with example usage: obj.text_to_instance(question_text, passage_text, token_spans, answer_texts, question_tokens, passage_tokens) and returns: util