class_docstr: Hidden Markov model class, a generative model for labelling sequence data.
  These models define the joint probability of a sequence of symbols and
  their labels (state transitions) as the product of the starting state
  probability, the probability of each state transition, and the probability
  of each observation being generated from each state. This is described in
  more detail in the module documentation.
  
  This implementation is based on the HMM description in Chapter 8, Huang,
  Acero and Hon, Spoken Language Processing and includes an extension for
  training shallow HMM parsers or specialized HMMs as in Molina et.
  al, 2002.  A specialized HMM modifies training data by applying a
  specialization function to create a new training set that is more
  appropriate for sequential tagging with an HMM.  A typical use case is
  chunking.
  
  :param symbols: the set of output symbols (alphabet)
  :type symbols: seq of any
  :param states: a set of states representing state space
  :type states: seq of any
  :param transitions: transition probabilities; Pr(s_i | s_j) is the
      probability of transition from state i given the model is in
      state_j
  :type transitions: ConditionalProbDistI
  :param outputs: output probabilities; Pr(o_k | s_i) is the probability
      of emitting symbol k when entering state i
  :type outputs: ConditionalProbDistI
  :param priors: initial state distribution; Pr(s_i) is the probability
      of starting in state i
  :type priors: ProbDistI
  :param transform: an optional function for transforming training
      instances, defaults to the identity function.
  :type transform: callable
docstr: Returns the entropy over labellings of the given sequence. This is
  given by::
  
      H(O) = - sum_S Pr(S | O) log Pr(S | O)
  
  where the summation ranges over all state sequences, S. Let
  *Z = Pr(O) = sum_S Pr(S, O)}* where the summation ranges over all state
  sequences and O is the observation sequence. As such the entropy can
  be re-expressed as::
  
      H = - sum_S Pr(S | O) log [ Pr(S, O) / Z ]
      = log Z - sum_S Pr(S | O) log Pr(S, 0)
      = log Z - sum_S Pr(S | O) [ log Pr(S_0) + sum_t Pr(S_t | S_{t-1}) + sum_t Pr(O_t | S_t) ]
  
  The order of summation for the log terms can be flipped, allowing
  dynamic programming to be used to calculate the entropy. Specifically,
  we use the forward and backward probabilities (alpha, beta) giving::
  
      H = log Z - sum_s0 alpha_0(s0) beta_0(s0) / Z * log Pr(s0)
      + sum_t,si,sj alpha_t(si) Pr(sj | si) Pr(O_t+1 | sj) beta_t(sj) / Z * log Pr(sj | si)
      + sum_t,st alpha_t(st) beta_t(st) / Z * log Pr(O_t | st)
  
  This simply uses alpha and beta to find the probabilities of partial
  sequences, constrained to include the given state(s) at some point in
  time.
docstr_tok: ["Returns","the","entropy","over","labellings","of","the","given","sequence",".","This","is","given","by",":",":","H","(","O",")","=","-","sum_S","Pr","(","S","|","O",")","log","Pr","(","S","|","O",")","where","the","summation","ranges","over","all","state","sequences",",","S.","Let","*","Z","=","Pr","(","O",")","=","sum_S","Pr","(","S",",","O",")","}","*","where","the","summation","ranges","over","all","state","sequences","and","O","is","the","observation","sequence",".","As","such","the","entropy","can","be","re-expressed","as",":",":","H","=","-","sum_S","Pr","(","S","|","O",")","log","[","Pr","(","S",",","O",")","\/","Z","]","=","log","Z","-","sum_S","Pr","(","S","|","O",")","log","Pr","(","S",",","0",")","=","log","Z","-","sum_S","Pr","(","S","|","O",")","[","log","Pr","(","S_0",")","+","sum_t","Pr","(","S_t","|","S_","{","t-1","}",")","+","sum_t","Pr","(","O_t","|","S_t",")","]","The","order","of","summation","for","the","log","terms","can","be","flipped",",","allowing","dynamic","programming","to","be","used","to","calculate","the","entropy",".","Specifically",",","we","use","the","forward","and","backward","probabilities","(","alpha",",","beta",")","giving",":",":","H","=","log","Z","-","sum_s0","alpha_0","(","s0",")","beta_0","(","s0",")","\/","Z","*","log","Pr","(","s0",")","+","sum_t",",","si",",","sj","alpha_t","(","si",")","Pr","(","sj","|","si",")","Pr","(","O_t+1","|","sj",")","beta_t","(","sj",")","\/","Z","*","log","Pr","(","sj","|","si",")","+","sum_t",",","st","alpha_t","(","st",")","beta_t","(","st",")","\/","Z","*","log","Pr","(","O_t","|","st",")","This","simply","uses","alpha","and","beta","to","find","the","probabilities","of","partial","sequences",",","constrained","to","include","the","given","state","(","s",")","at","some","point","in","time","."]
code: def entropy(self, unlabeled_sequence):
      """
      Returns the entropy over labellings of the given sequence. This is
      given by::
  
          H(O) = - sum_S Pr(S | O) log Pr(S | O)
  
      where the summation ranges over all state sequences, S. Let
      *Z = Pr(O) = sum_S Pr(S, O)}* where the summation ranges over all state
      sequences and O is the observation sequence. As such the entropy can
      be re-expressed as::
  
          H = - sum_S Pr(S | O) log [ Pr(S, O) / Z ]
          = log Z - sum_S Pr(S | O) log Pr(S, 0)
          = log Z - sum_S Pr(S | O) [ log Pr(S_0) + sum_t Pr(S_t | S_{t-1}) + sum_t Pr(O_t | S_t) ]
  
      The order of summation for the log terms can be flipped, allowing
      dynamic programming to be used to calculate the entropy. Specifically,
      we use the forward and backward probabilities (alpha, beta) giving::
  
          H = log Z - sum_s0 alpha_0(s0) beta_0(s0) / Z * log Pr(s0)
          + sum_t,si,sj alpha_t(si) Pr(sj | si) Pr(O_t+1 | sj) beta_t(sj) / Z * log Pr(sj | si)
          + sum_t,st alpha_t(st) beta_t(st) / Z * log Pr(O_t | st)
  
      This simply uses alpha and beta to find the probabilities of partial
      sequences, constrained to include the given state(s) at some point in
      time.
      """
      unlabeled_sequence = self._transform(unlabeled_sequence)
  
      T = len(unlabeled_sequence)
      N = len(self._states)
  
      alpha = self._forward_probability(unlabeled_sequence)
      beta = self._backward_probability(unlabeled_sequence)
      normalisation = logsumexp2(alpha[T - 1])
  
      entropy = normalisation
  
      # starting state, t = 0
      for i, state in enumerate(self._states):
          p = 2 ** (alpha[0, i] + beta[0, i] - normalisation)
          entropy -= p * self._priors.logprob(state)
          # print('p(s_0 = %s) =' % state, p)
  
      # state transitions
      for t0 in range(T - 1):
          t1 = t0 + 1
          for i0, s0 in enumerate(self._states):
              for i1, s1 in enumerate(self._states):
                  p = 2 ** (
                      alpha[t0, i0]
                      + self._transitions[s0].logprob(s1)
                      + self._outputs[s1].logprob(
                          unlabeled_sequence[t1][_TEXT]
                      )
                      + beta[t1, i1]
                      - normalisation
                  )
                  entropy -= p * self._transitions[s0].logprob(s1)
                  # print('p(s_%d = %s, s_%d = %s) =' % (t0, s0, t1, s1), p)
  
      # symbol emissions
      for t in range(T):
          for i, state in enumerate(self._states):
              p = 2 ** (alpha[t, i] + beta[t, i] - normalisation)
              entropy -= p * self._outputs[state].logprob(
                  unlabeled_sequence[t][_TEXT]
              )
              # print('p(s_%d = %s) =' % (t, state), p)
  
      return entropy
  
imports: import itertools
  import re
  from nltk.metrics import accuracy
  from nltk.probability import ConditionalFreqDist, ConditionalProbDist, DictionaryConditionalProbDist, DictionaryProbDist, FreqDist, LidstoneProbDist, MLEProbDist, MutableProbDist, RandomProbDist
  from nltk.tag.api import TaggerI
  from nltk.util import LazyMap, unique_list
label_desc: Use image node_id 18 for calling the HiddenMarkovModelTagger obj's underlying member method code with example usage: obj.entropy(unlabeled_sequence) and returns: entropy