class_docstr: This is a simple class of models that tags individual words in text.
code: def _get_data_points_from_sentence(
      self, sentence: Sentence
  ) -> List[Token]:
      # special handling during training if this is a span prediction problem
      if self.training and self.span_prediction_problem:
          for token in sentence.tokens:
              token.set_label(self.label_type, "O")
              for span in sentence.get_spans(self.label_type):
                  span_label = span.get_label(self.label_type).value
                  if len(span) == 1:
                      if self.span_encoding == "BIOES":
                          span.tokens[0].set_label(
                              self.label_type, "S-" + span_label
                          )
                      elif self.span_encoding == "BIO":
                          span.tokens[0].set_label(
                              self.label_type, "B-" + span_label
                          )
                  else:
                      for token in span.tokens:
                          token.set_label(
                              self.label_type, "I-" + span_label
                          )
                      span.tokens[0].set_label(
                          self.label_type, "B-" + span_label
                      )
                      if self.span_encoding == "BIOES":
                          span.tokens[-1].set_label(
                              self.label_type, "E-" + span_label
                          )
  
      return sentence.tokens
  
imports: import logging
  from pathlib import Path
  from typing import Any, Dict, List, Union
  import torch
  from deprecated.sphinx import deprecated
  import flair.nn
  from flair.data import Dictionary, Sentence, Span, Token
  from flair.embeddings import TokenEmbeddings
label_desc: Use image node_id 7 for calling the TokenClassifier obj's underlying member method code with example usage: obj._get_data_points_from_sentence(sentence) and returns: sentence