class_docstr: This DatasetReader is designed to read in the English OntoNotes v5.0 data
  in the format used by the CoNLL 2011/2012 shared tasks. In order to use this
  Reader, you must follow the instructions provided `here (v12 release):
  <http://cemantix.org/data/ontonotes.html>`_, which will allow you to download
  the CoNLL style annotations for the  OntoNotes v5.0 release -- LDC2013T19.tgz
  obtained from LDC.
  
  Once you have run the scripts on the extracted data, you will have a folder
  structured as follows:
  
  conll-formatted-ontonotes-5.0/
   ── data
     ├── development
         └── data
             └── english
                 └── annotations
                     ├── bc
                     ├── bn
                     ├── mz
                     ├── nw
                     ├── pt
                     ├── tc
                     └── wb
     ├── test
         └── data
             └── english
                 └── annotations
                     ├── bc
                     ├── bn
                     ├── mz
                     ├── nw
                     ├── pt
                     ├── tc
                     └── wb
     └── train
         └── data
             └── english
                 └── annotations
                     ├── bc
                     ├── bn
                     ├── mz
                     ├── nw
                     ├── pt
                     ├── tc
                     └── wb
  
  The file path provided to this class can then be any of the train, test or development
  directories(or the top level data directory, if you are not utilizing the splits).
  
  The data has the following format, ordered by column.
  
  1 Document ID : ``str``
      This is a variation on the document filename
  2 Part number : ``int``
      Some files are divided into multiple parts numbered as 000, 001, 002, ... etc.
  3 Word number : ``int``
      This is the word index of the word in that sentence.
  4 Word : ``str``
      This is the token as segmented/tokenized in the Treebank. Initially the ``*_skel`` file
      contain the placeholder [WORD] which gets replaced by the actual token from the
      Treebank which is part of the OntoNotes release.
  5 POS Tag : ``str``
      This is the Penn Treebank style part of speech. When parse information is missing,
      all part of speeches except the one for which there is some sense or proposition
      annotation are marked with a XX tag. The verb is marked with just a VERB tag.
  6 Parse bit: ``str``
      This is the bracketed structure broken before the first open parenthesis in the parse,
      and the word/part-of-speech leaf replaced with a ``*``. When the parse information is
      missing, the first word of a sentence is tagged as ``(TOP*`` and the last word is tagged
      as ``*)`` and all intermediate words are tagged with a ``*``.
  7 Predicate lemma: ``str``
      The predicate lemma is mentioned for the rows for which we have semantic role
      information or word sense information. All other rows are marked with a "-".
  8 Predicate Frameset ID: ``int``
      The PropBank frameset ID of the predicate in Column 7.
  9 Word sense: ``float``
      This is the word sense of the word in Column 3.
  10 Speaker/Author: ``str``
      This is the speaker or author name where available. Mostly in Broadcast Conversation
      and Web Log data. When not available the rows are marked with an "-".
  11 Named Entities: ``str``
      These columns identifies the spans representing various named entities. For documents
      which do not have named entity annotation, each line is represented with an ``*``.
  12+ Predicate Arguments: ``str``
      There is one column each of predicate argument structure information for the predicate
      mentioned in Column 7. If there are no predicates tagged in a sentence this is a
      single column with all rows marked with an ``*``.
  -1 Co-reference: ``str``
      Co-reference chain information encoded in a parenthesis structure. For documents that do
       not have co-reference annotations, each line is represented with a "-".
class_docstr_tok: ["This","DatasetReader","is","designed","to","read","in","the","English","OntoNotes","v5.0","data","in","the","format","used","by","the","CoNLL","2011\/2012","shared","tasks",".","In","order","to","use","this","Reader",",","you","must","follow","the","instructions","provided","`","here","(","v12","release",")",":","<","http",":","\/\/cemantix.org\/data\/ontonotes.html",">","`","_",",","which","will","allow","you","to","download","the","CoNLL","style","annotations","for","the","OntoNotes","v5.0","release","--","LDC2013T19.tgz","obtained","from","LDC",".","Once","you","have","run","the","scripts","on","the","extracted","data",",","you","will","have","a","folder","structured","as","follows",":","conll-formatted-ontonotes-5.0\/","\u2500\u2500","data","\u251c\u2500\u2500","development","\u2514\u2500\u2500","data","\u2514\u2500\u2500","english","\u2514\u2500\u2500","annotations","\u251c\u2500\u2500","bc","\u251c\u2500\u2500","bn","\u251c\u2500\u2500","mz","\u251c\u2500\u2500","nw","\u251c\u2500\u2500","pt","\u251c\u2500\u2500","tc","\u2514\u2500\u2500","wb","\u251c\u2500\u2500","test","\u2514\u2500\u2500","data","\u2514\u2500\u2500","english","\u2514\u2500\u2500","annotations","\u251c\u2500\u2500","bc","\u251c\u2500\u2500","bn","\u251c\u2500\u2500","mz","\u251c\u2500\u2500","nw","\u251c\u2500\u2500","pt","\u251c\u2500\u2500","tc","\u2514\u2500\u2500","wb","\u2514\u2500\u2500","train","\u2514\u2500\u2500","data","\u2514\u2500\u2500","english","\u2514\u2500\u2500","annotations","\u251c\u2500\u2500","bc","\u251c\u2500\u2500","bn","\u251c\u2500\u2500","mz","\u251c\u2500\u2500","nw","\u251c\u2500\u2500","pt","\u251c\u2500\u2500","tc","\u2514\u2500\u2500","wb","The","file","path","provided","to","this","class","can","then","be","any","of","the","train",",","test","or","development","directories","(","or","the","top","level","data","directory",",","if","you","are","not","utilizing","the","splits",")",".","The","data","has","the","following","format",",","ordered","by","column",".","1","Document","ID",":","``","str","``","This","is","a","variation","on","the","document","filename","2","Part","number",":","``","int","``","Some","files","are","divided","into","multiple","parts","numbered","as","000",",","001",",","002",",","...","etc",".","3","Word","number",":","``","int","``","This","is","the","word","index","of","the","word","in","that","sentence",".","4","Word",":","``","str","``","This","is","the","token","as","segmented\/tokenized","in","the","Treebank",".","Initially","the","``","*","_skel","``","file","contain","the","placeholder","[","WORD","]","which","gets","replaced","by","the","actual","token","from","the","Treebank","which","is","part","of","the","OntoNotes","release",".","5","POS","Tag",":","``","str","``","This","is","the","Penn","Treebank","style","part","of","speech",".","When","parse","information","is","missing",",","all","part","of","speeches","except","the","one","for","which","there","is","some","sense","or","proposition","annotation","are","marked","with","a","XX","tag",".","The","verb","is","marked","with","just","a","VERB","tag",".","6","Parse","bit",":","``","str","``","This","is","the","bracketed","structure","broken","before","the","first","open","parenthesis","in","the","parse",",","and","the","word\/part-of-speech","leaf","replaced","with","a","``","*","``",".","When","the","parse","information","is","missing",",","the","first","word","of","a","sentence","is","tagged","as","``","(","TOP","*","``","and","the","last","word","is","tagged","as","``","*",")","``","and","all","intermediate","words","are","tagged","with","a","``","*","``",".","7","Predicate","lemma",":","``","str","``","The","predicate","lemma","is","mentioned","for","the","rows","for","which","we","have","semantic","role","information","or","word","sense","information",".","All","other","rows","are","marked","with","a","``","-","''",".","8","Predicate","Frameset","ID",":","``","int","``","The","PropBank","frameset","ID","of","the","predicate","in","Column","7",".","9","Word","sense",":","``","float","``","This","is","the","word","sense","of","the","word","in","Column","3",".","10","Speaker\/Author",":","``","str","``","This","is","the","speaker","or","author","name","where","available",".","Mostly","in","Broadcast","Conversation","and","Web","Log","data",".","When","not","available","the","rows","are","marked","with","an","``","-","''",".","11","Named","Entities",":","``","str","``","These","columns","identifies","the","spans","representing","various","named","entities",".","For","documents","which","do","not","have","named","entity","annotation",",","each","line","is","represented","with","an","``","*","``",".","12+","Predicate","Arguments",":","``","str","``","There","is","one","column","each","of","predicate","argument","structure","information","for","the","predicate","mentioned","in","Column","7",".","If","there","are","no","predicates","tagged","in","a","sentence","this","is","a","single","column","with","all","rows","marked","with","an","``","*","``",".","-1","Co-reference",":","``","str","``","Co-reference","chain","information","encoded","in","a","parenthesis","structure",".","For","documents","that","do","not","have","co-reference","annotations",",","each","line","is","represented","with","a","``","-","''","."]
docstr: An iterator over the entire dataset, yielding all sentences processed.
docstr_tok: ["An","iterator","over","the","entire","dataset",",","yielding","all","sentences","processed","."]
code: def dataset_iterator(self, file_path):
      """
      An iterator over the entire dataset, yielding all sentences processed.
      """
      for conll_file in self.dataset_path_iterator(file_path):
          yield self.sentence_iterator(conll_file)
  
file_path: magnitude/pymagnitude/third_party/allennlp/data/dataset_readers/dataset_utils/ontonotes.py
imports: from __future__ import with_statement
  from __future__ import absolute_import
  from collections import defaultdict
  import codecs
  import os
  import logging
  from nltk import Tree
label_desc: Use image node_id 1 for calling the Ontonotes obj's underlying member method code with example usage: obj.dataset_iterator(file_path) without return types