class_docstr: This class loads in the Facebook bAbI dataset and vectorizes them into stories,
  questions, and answers as described in:
  "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"
  http://arxiv.org/abs/1502.05698.
class_docstr_tok: ["This","class","loads","in","the","Facebook","bAbI","dataset","and","vectorizes","them","into","stories",",","questions",",","and","answers","as","described","in",":","''","Towards","AI-Complete","Question","Answering",":","A","Set","of","Prerequisite","Toy","Tasks","''","http",":","\/\/arxiv.org\/abs\/1502.05698","."]
docstr: Compute vocab, word index, and max length of stories and queries.
docstr_tok: ["Compute","vocab",",","word","index",",","and","max","length","of","stories","and","queries","."]
code: def compute_statistics(self):
      """
      Compute vocab, word index, and max length of stories and queries.
      """
      all_data = self.train_parsed + self.test_parsed
      vocab = sorted(
          reduce(
              lambda x, y: x | y,
              (set(s + q + [a]) for s, q, a in all_data),
          )
      )
      self.vocab = vocab
      # Reserve 0 for masking via pad_sequences and self.vocab_size - 1 for <UNK> token
      self.vocab_size = len(vocab) + 2
      self.word_to_index = dict((c, i + 1) for i, c in enumerate(vocab))
      self.index_to_word = dict((i + 1, c) for i, c in enumerate(vocab))
      self.story_maxlen = max(
          list(map(len, (s for s, _, _ in all_data)))
      )
      self.query_maxlen = max(
          list(map(len, (q for _, q, _ in all_data)))
      )
  
imports: from builtins import map
  import numpy
  import os
  import re
  import tarfile
  from neon import logger
  from neon.data.dataiterator import NervanaDataIterator
  from neon.data.datasets import Dataset
  from neon.data.text_preprocessing import pad_sentences
  from functools import reduce
label_desc: Use image node_id 10 for calling the BABI obj's underlying member method code with example usage: obj.compute_statistics() without return types