class_docstr: Reads instances from a pretokenised file where each line is in the following format:
  
  WORD POS-TAG CHUNK-TAG NER-TAG
  
  with a blank line indicating the end of each sentence
  and '-DOCSTART- -X- -X- O' indicating the end of each article,
  and converts it into a ``Dataset`` suitable for sequence tagging.
  
  Each ``Instance`` contains the words in the ``"tokens"`` ``TextField``.
  The values corresponding to the ``tag_label``
  values will get loaded into the ``"tags"`` ``SequenceLabelField``.
  And if you specify any ``feature_labels`` (you probably shouldn't),
  the corresponding values will get loaded into their own ``SequenceLabelField`` s.
  
  This dataset reader ignores the "article" divisions and simply treats
  each sentence as an independent ``Instance``. (Technically the reader splits sentences
  on any combination of blank lines and "DOCSTART" tags; in particular, it does the right
  thing on well formed inputs.)
  
  Parameters
  ----------
  token_indexers : ``Dict[str, TokenIndexer]``, optional (default=``{"tokens": SingleIdTokenIndexer()}``)
      We use this to define the input representation for the text.  See :class:`TokenIndexer`.
  tag_label: ``str``, optional (default=``ner``)
      Specify `ner`, `pos`, or `chunk` to have that tag loaded into the instance field `tag`.
  feature_labels: ``Sequence[str]``, optional (default=``()``)
      These labels will be loaded as features into the corresponding instance fields:
      ``pos`` -> ``pos_tags``, ``chunk`` -> ``chunk_tags``, ``ner`` -> ``ner_tags``
      Each will have its own namespace: ``pos_labels``, ``chunk_labels``, ``ner_labels``.
      If you want to use one of the labels as a `feature` in your model, it should be
      specified here.
  coding_scheme: ``str``, optional (default=``IOB1``)
      Specifies the coding scheme for ``ner_labels`` and ``chunk_labels``.
      Valid options are ``IOB1`` and ``BIOUL``.  The ``IOB1`` default maintains
      the original IOB1 scheme in the CoNLL data.
      In the IOB1 scheme, I is a token inside a span, O is a token outside
      a span and B is the beginning of span immediately following another
      span of the same type.
class_docstr_tok: ["Reads","instances","from","a","pretokenised","file","where","each","line","is","in","the","following","format",":","WORD","POS-TAG","CHUNK-TAG","NER-TAG","with","a","blank","line","indicating","the","end","of","each","sentence","and","'-DOCSTART-","-X-","-X-","O","'","indicating","the","end","of","each","article",",","and","converts","it","into","a","``","Dataset","``","suitable","for","sequence","tagging",".","Each","``","Instance","``","contains","the","words","in","the","``","``","tokens","''","``","``","TextField","``",".","The","values","corresponding","to","the","``","tag_label","``","values","will","get","loaded","into","the","``","``","tags","''","``","``","SequenceLabelField","``",".","And","if","you","specify","any","``","feature_labels","``","(","you","probably","should","n't",")",",","the","corresponding","values","will","get","loaded","into","their","own","``","SequenceLabelField","``","s.","This","dataset","reader","ignores","the","``","article","''","divisions","and","simply","treats","each","sentence","as","an","independent","``","Instance","``",".","(","Technically","the","reader","splits","sentences","on","any","combination","of","blank","lines","and","``","DOCSTART","''","tags",";","in","particular",",","it","does","the","right","thing","on","well","formed","inputs",".",")","Parameters","--","--","--","--","--","token_indexers",":","``","Dict","[","str",",","TokenIndexer","]","``",",","optional","(","default=","``","{","``","tokens","''",":","SingleIdTokenIndexer","(",")","}","``",")","We","use","this","to","define","the","input","representation","for","the","text",".","See",":","class",":","`","TokenIndexer","`",".","tag_label",":","``","str","``",",","optional","(","default=","``","ner","``",")","Specify","`","ner","`",",","`","pos","`",",","or","`","chunk","`","to","have","that","tag","loaded","into","the","instance","field","`","tag","`",".","feature_labels",":","``","Sequence","[","str","]","``",",","optional","(","default=","``","(",")","``",")","These","labels","will","be","loaded","as","features","into","the","corresponding","instance","fields",":","``","pos","``","-",">","``","pos_tags","``",",","``","chunk","``","-",">","``","chunk_tags","``",",","``","ner","``","-",">","``","ner_tags","``","Each","will","have","its","own","namespace",":","``","pos_labels","``",",","``","chunk_labels","``",",","``","ner_labels","``",".","If","you","want","to","use","one","of","the","labels","as","a","`","feature","`","in","your","model",",","it","should","be","specified","here",".","coding_scheme",":","``","str","``",",","optional","(","default=","``","IOB1","``",")","Specifies","the","coding","scheme","for","``","ner_labels","``","and","``","chunk_labels","``",".","Valid","options","are","``","IOB1","``","and","``","BIOUL","``",".","The","``","IOB1","``","default","maintains","the","original","IOB1","scheme","in","the","CoNLL","data",".","In","the","IOB1","scheme",",","I","is","a","token","inside","a","span",",","O","is","a","token","outside","a","span","and","B","is","the","beginning","of","span","immediately","following","another","span","of","the","same","type","."]
code: def __init__(
      self,
      token_indexers=None,
      tag_label="ner",
      feature_labels=(),
      lazy=False,
      coding_scheme="IOB1",
  ):
      super(Conll2003DatasetReader, self).__init__(lazy)
      self._token_indexers = token_indexers or {
          "tokens": SingleIdTokenIndexer()
      }
      if tag_label is not None and tag_label not in _VALID_LABELS:
          raise ConfigurationError(
              "unknown tag label type: {}".format(tag_label)
          )
      for label in feature_labels:
          if label not in _VALID_LABELS:
              raise ConfigurationError(
                  "unknown feature label type: {}".format(label)
              )
      if coding_scheme not in ("IOB1", "BIOUL"):
          raise ConfigurationError(
              "unknown coding_scheme: {}".format(coding_scheme)
          )
  
      self.tag_label = tag_label
      self.feature_labels = set(feature_labels)
      self.coding_scheme = coding_scheme
  
file_path: magnitude/pymagnitude/third_party/allennlp/data/dataset_readers/conll2003.py
imports: from __future__ import with_statement
  from __future__ import absolute_import
  import itertools
  import logging
  from allennlp.common.checks import ConfigurationError
  from allennlp.common.file_utils import cached_path
  from allennlp.data.dataset_readers.dataset_reader import DatasetReader
  from allennlp.data.dataset_readers.dataset_utils import to_bioul
  from allennlp.data.fields import TextField, SequenceLabelField, Field, MetadataField
  from allennlp.data.instance import Instance
  from allennlp.data.token_indexers import TokenIndexer, SingleIdTokenIndexer
  from allennlp.data.tokenizers import Token
  from io import open
label_desc: Use image node_id 1 to create a new Conll2003DatasetReader object from inherited base classes: DatasetReader with example: obj = Conll2003DatasetReader(token_indexers, tag_label, feature_labels, lazy, coding_scheme)