class_docstr: An implementation of linear chain conditional random field (CRF).
  An linear chain CRF is defined to maximize the following likelihood function:
  $$ L(W, U, b; y_1, ..., y_n) := rac{1}{Z} \sum_{y_1, ..., y_n} \exp(-a_1' y_1 - a_n' y_n
      - \sum_{k=1^n}((f(x_k' W + b) y_k) + y_1' U y_2)), $$
  where:
      $Z$: normalization constant
      $x_k, y_k$:  inputs and outputs
  This implementation has two modes for optimization:
  1. (`join mode`) optimized by maximizing join likelihood, which is optimal in theory of statistics.
     Note that in this case, CRF must be the output/last layer.
  2. (`marginal mode`) return marginal probabilities on each time step and optimized via composition
     likelihood (product of marginal likelihood), i.e., using `categorical_crossentropy` loss.
     Note that in this case, CRF can be either the last layer or an intermediate layer (though not explored).
  For prediction (test phrase), one can choose either Viterbi best path (class indices) or marginal
  probabilities if probabilities are needed. However, if one chooses *join mode* for training,
  Viterbi output is typically better than marginal output, but the marginal output will still perform
  reasonably close, while if *marginal mode* is used for training, marginal output usually performs
  much better. The default behavior is set according to this observation.
  In addition, this implementation supports masking and accepts either onehot or sparse target.
  # Examples
  ```python
      model = Sequential()
      model.add(Embedding(3001, 300, mask_zero=True)(X)
      # use learn_mode = 'join', test_mode = 'viterbi', sparse_target = True (label indice output)
      crf = CRF(10, sparse_target=True)
      model.add(crf)
      # crf.accuracy is default to Viterbi acc if using join-mode (default).
      # One can add crf.marginal_acc if interested, but may slow down learning
      model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
      # y must be label indices (with shape 1 at dim 3) here, since `sparse_target=True`
      model.fit(x, y)
      # prediction give onehot representation of Viterbi best path
      y_hat = model.predict(x_test)
  ```
  # Arguments
      units: Positive integer, dimensionality of the output space.
      learn_mode: Either 'join' or 'marginal'.
          The former train the model by maximizing join likelihood while the latter
          maximize the product of marginal likelihood over all time steps.
      test_mode: Either 'viterbi' or 'marginal'.
          The former is recommended and as default when `learn_mode = 'join'` and
          gives one-hot representation of the best path at test (prediction) time,
          while the latter is recommended and chosen as default when `learn_mode = 'marginal'`,
          which produces marginal probabilities for each time step.
      sparse_target: Boolean (default False) indicating if provided labels are one-hot or
          indices (with shape 1 at dim 3).
      use_boundary: Boolean (default True) indicating if trainable start-end chain energies
          should be added to model.
      use_bias: Boolean, whether the layer uses a bias vector.
      kernel_initializer: Initializer for the `kernel` weights matrix,
          used for the linear transformation of the inputs.
          (see [initializers](../initializers.md)).
      chain_initializer: Initializer for the `chain_kernel` weights matrix,
          used for the CRF chain energy.
          (see [initializers](../initializers.md)).
      boundary_initializer: Initializer for the `left_boundary`, 'right_boundary' weights vectors,
          used for the start/left and end/right boundary energy.
          (see [initializers](../initializers.md)).
      bias_initializer: Initializer for the bias vector
          (see [initializers](../initializers.md)).
      activation: Activation function to use
          (see [activations](../activations.md)).
          If you pass None, no activation is applied
          (ie. "linear" activation: `a(x) = x`).
      kernel_regularizer: Regularizer function applied to
          the `kernel` weights matrix
          (see [regularizer](../regularizers.md)).
      chain_regularizer: Regularizer function applied to
          the `chain_kernel` weights matrix
          (see [regularizer](../regularizers.md)).
      boundary_regularizer: Regularizer function applied to
          the 'left_boundary', 'right_boundary' weight vectors
          (see [regularizer](../regularizers.md)).
      bias_regularizer: Regularizer function applied to the bias vector
          (see [regularizer](../regularizers.md)).
      kernel_constraint: Constraint function applied to
          the `kernel` weights matrix
          (see [constraints](../constraints.md)).
      chain_constraint: Constraint function applied to
          the `chain_kernel` weights matrix
          (see [constraints](../constraints.md)).
      boundary_constraint: Constraint function applied to
          the `left_boundary`, `right_boundary` weights vectors
          (see [constraints](../constraints.md)).
      bias_constraint: Constraint function applied to the bias vector
          (see [constraints](../constraints.md)).
      input_dim: dimensionality of the input (integer).
          This argument (or alternatively, the keyword argument `input_shape`)
          is required when using this layer as the first layer in a model.
      unroll: Boolean (default False). If True, the network will be unrolled, else a symbolic loop will be used.
          Unrolling can speed-up a RNN, although it tends to be more memory-intensive.
          Unrolling is only suitable for short sequences.
  # Input shape
      3D tensor with shape `(nb_samples, timesteps, input_dim)`.
  # Output shape
      3D tensor with shape `(nb_samples, timesteps, units)`.
  # Masking
      This layer supports masking for input data with a variable number
      of timesteps. To introduce masks to your data,
      use an [Embedding](embeddings.md) layer with the `mask_zero` parameter
      set to `True`.
code: def step(self, input_energy_t, states, return_logZ=True):
      # not in the following  `prev_target_val` has shape = (B, F)
      # where B = batch_size, F = output feature dim
      # Note: `i` is of float32, due to the behavior of `K.rnn`
      prev_target_val, i, chain_energy = states[:3]
      t = K.cast(i[0, 0], dtype="int32")
      if len(states) > 3:
          if K.backend() == "theano":
              m = states[3][:, t : (t + 2)]
          else:
              m = K.tf.slice(states[3], [0, t], [-1, 2])
          input_energy_t = input_energy_t * K.expand_dims(m[:, 0])
          chain_energy = chain_energy * K.expand_dims(
              K.expand_dims(m[:, 0] * m[:, 1])
          )  # (1, F, F)*(B, 1, 1) -> (B, F, F)
      if return_logZ:
          energy = chain_energy + K.expand_dims(
              input_energy_t - prev_target_val, 2
          )  # shapes: (1, B, F) + (B, F, 1) -> (B, F, F)
          new_target_val = K.logsumexp(-energy, 1)  # shapes: (B, F)
          return new_target_val, [new_target_val, i + 1]
      else:
          energy = chain_energy + K.expand_dims(
              input_energy_t + prev_target_val, 2
          )
          min_energy = K.min(energy, 1)
          argmin_table = K.cast(
              K.argmin(energy, 1), K.floatx()
          )  # cast for tf-version `K.rnn`
          return argmin_table, [min_energy, i + 1]
  
imports: from __future__ import absolute_import
  from __future__ import division
  import keras.backend
  from keras import activations
  from keras import initializers
  from keras import regularizers
  from keras import constraints
  from keras.engine import Layer
  from keras.engine import InputSpec
  from keras.objectives import categorical_crossentropy
  from keras.objectives import sparse_categorical_crossentropy
label_desc: Use image node_id 19 for calling the CRF obj's underlying member method code with example usage: obj.step(input_energy_t, states, return_logZ) and returns: new_target_val,  list, argmin_table,  list