class_docstr: A ``WikiTablesSemanticParser`` is a :class:`Model` which takes as input a table and a question,
  and produces a logical form that answers the question when executed over the table.  The
  logical form is generated by a `type-constrained`, `transition-based` parser. This is an
  abstract class that defines most of the functionality related to the transition-based parser. It
  does not contain the implementation for actually training the parser. You may want to train it
  using a learning-to-search algorithm, in which case you will want to use
  ``WikiTablesErmSemanticParser``, or if you have a set of approximate logical forms that give the
  correct denotation, you will want to use ``WikiTablesMmlSemanticParser``.
  
  Parameters
  ----------
  vocab : ``Vocabulary``
  question_embedder : ``TextFieldEmbedder``
      Embedder for questions.
  action_embedding_dim : ``int``
      Dimension to use for action embeddings.
  encoder : ``Seq2SeqEncoder``
      The encoder to use for the input question.
  entity_encoder : ``Seq2VecEncoder``
      The encoder to used for averaging the words of an entity.
  max_decoding_steps : ``int``
      When we're decoding with a beam search, what's the maximum number of steps we should take?
      This only applies at evaluation time, not during training.
  use_neighbor_similarity_for_linking : ``bool``, optional (default=False)
      If ``True``, we will compute a max similarity between a question token and the `neighbors`
      of an entity as a component of the linking scores.  This is meant to capture the same kind
      of information as the ``related_column`` feature.
  dropout : ``float``, optional (default=0)
      If greater than 0, we will apply dropout with this probability after all encoders (pytorch
      LSTMs do not apply dropout to their last layer).
  num_linking_features : ``int``, optional (default=10)
      We need to construct a parameter vector for the linking features, so we need to know how
      many there are.  The default of 8 here matches the default in the ``KnowledgeGraphField``,
      which is to use all eight defined features. If this is 0, another term will be added to the
      linking score. This term contains the maximum similarity value from the entity's neighbors
      and the question.
  rule_namespace : ``str``, optional (default=rule_labels)
      The vocabulary namespace to use for production rules.  The default corresponds to the
      default used in the dataset reader, so you likely don't need to modify this.
  tables_directory : ``str``, optional (default=/wikitables/)
      The directory to find tables when evaluating logical forms.  We rely on a call to SEMPRE to
      evaluate logical forms, and SEMPRE needs to read the table from disk itself.  This tells
      SEMPRE where to find the tables.
docstr: Does initial preparation and creates an intiial state for both the semantic parsers. Note
  that the checklist state is optional, and the ``WikiTablesMmlParser`` is not expected to
  pass it.
docstr_tok: ["Does","initial","preparation","and","creates","an","intiial","state","for","both","the","semantic","parsers",".","Note","that","the","checklist","state","is","optional",",","and","the","``","WikiTablesMmlParser","``","is","not","expected","to","pass","it","."]
code: def _get_initial_state_and_scores(
      self,
      question,
      table,
      world,
      actions,
      example_lisp_string=None,
      add_world_to_initial_state=False,
      checklist_states=None,
  ):
      """
      Does initial preparation and creates an intiial state for both the semantic parsers. Note
      that the checklist state is optional, and the ``WikiTablesMmlParser`` is not expected to
      pass it.
      """
      table_text = table["text"]
      # (batch_size, question_length, embedding_dim)
      embedded_question = self._question_embedder(question)
      question_mask = util.get_text_field_mask(question).float()
      # (batch_size, num_entities, num_entity_tokens, embedding_dim)
      embedded_table = self._question_embedder(
          table_text, num_wrapping_dims=1
      )
      table_mask = util.get_text_field_mask(
          table_text, num_wrapping_dims=1
      ).float()
  
      (
          batch_size,
          num_entities,
          num_entity_tokens,
          _,
      ) = embedded_table.size()
      num_question_tokens = embedded_question.size(1)
  
      # (batch_size, num_entities, embedding_dim)
      encoded_table = self._entity_encoder(embedded_table, table_mask)
      # (batch_size, num_entities, num_neighbors)
      neighbor_indices = self._get_neighbor_indices(
          world, num_entities, encoded_table
      )
  
      # Neighbor_indices is padded with -1 since 0 is a potential neighbor index.
      # Thus, the absolute value needs to be taken in the index_select, and 1 needs to
      # be added for the mask since that method expects 0 for padding.
      # (batch_size, num_entities, num_neighbors, embedding_dim)
      embedded_neighbors = util.batched_index_select(
          encoded_table, torch.abs(neighbor_indices)
      )
  
      neighbor_mask = util.get_text_field_mask(
          {"ignored": neighbor_indices + 1}, num_wrapping_dims=1
      ).float()
  
      # Encoder initialized to easily obtain a masked average.
      neighbor_encoder = TimeDistributed(
          BagOfEmbeddingsEncoder(self._embedding_dim, averaged=True)
      )
      # (batch_size, num_entities, embedding_dim)
      embedded_neighbors = neighbor_encoder(
          embedded_neighbors, neighbor_mask
      )
  
      # entity_types: one-hot tensor with shape (batch_size, num_entities, num_types)
      # entity_type_dict: Dict[int, int], mapping flattened_entity_index -> type_index
      # These encode the same information, but for efficiency reasons later it's nice
      # to have one version as a tensor and one that's accessible on the cpu.
      entity_types, entity_type_dict = self._get_type_vector(
          world, num_entities, encoded_table
      )
  
      entity_type_embeddings = self._type_params(entity_types.float())
      projected_neighbor_embeddings = self._neighbor_params(
          embedded_neighbors.float()
      )
      # (batch_size, num_entities, embedding_dim)
      entity_embeddings = torch.tanh(
          entity_type_embeddings + projected_neighbor_embeddings
      )
  
      # Compute entity and question word similarity.  We tried using cosine distance here, but
      # because this similarity is the main mechanism that the model can use to push apart logit
      # scores for certain actions (like "n -> 1" and "n -> -1"), this needs to have a larger
      # output range than [-1, 1].
      question_entity_similarity = torch.bmm(
          embedded_table.view(
              batch_size,
              num_entities * num_entity_tokens,
              self._embedding_dim,
          ),
          torch.transpose(embedded_question, 1, 2),
      )
  
      question_entity_similarity = question_entity_similarity.view(
          batch_size,
          num_entities,
          num_entity_tokens,
          num_question_tokens,
      )
  
      # (batch_size, num_entities, num_question_tokens)
      question_entity_similarity_max_score, _ = torch.max(
          question_entity_similarity, 2
      )
  
      # (batch_size, num_entities, num_question_tokens, num_features)
      linking_features = table["linking"]
  
      linking_scores = question_entity_similarity_max_score
  
      if self._use_neighbor_similarity_for_linking:
          # The linking score is computed as a linear projection of two terms. The first is the
          # maximum similarity score over the entity's words and the question token. The second
          # is the maximum similarity over the words in the entity's neighbors and the question
          # token.
          #
          # The second term, projected_question_neighbor_similarity, is useful when a column
          # needs to be selected. For example, the question token might have no similarity with
          # the column name, but is similar with the cells in the column.
          #
          # Note that projected_question_neighbor_similarity is intended to capture the same
          # information as the related_column feature.
          #
          # Also note that this block needs to be _before_ the `linking_params` block, because
          # we're overwriting `linking_scores`, not adding to it.
  
          # (batch_size, num_entities, num_neighbors, num_question_tokens)
          question_neighbor_similarity = util.batched_index_select(
              question_entity_similarity_max_score,
              torch.abs(neighbor_indices),
          )
          # (batch_size, num_entities, num_question_tokens)
          question_neighbor_similarity_max_score, _ = torch.max(
              question_neighbor_similarity, 2
          )
          projected_question_entity_similarity = (
              self._question_entity_params(
                  question_entity_similarity_max_score.unsqueeze(-1)
              ).squeeze(-1)
          )
          projected_question_neighbor_similarity = (
              self._question_neighbor_params(
                  question_neighbor_similarity_max_score.unsqueeze(-1)
              ).squeeze(-1)
          )
          linking_scores = (
              projected_question_entity_similarity
              + projected_question_neighbor_similarity
          )
  
      feature_scores = None
      if self._linking_params is not None:
          feature_scores = self._linking_params(
              linking_features
          ).squeeze(3)
          linking_scores = linking_scores + feature_scores
  
      # (batch_size, num_question_tokens, num_entities)
      linking_probabilities = self._get_linking_probabilities(
          world,
          linking_scores.transpose(1, 2),
          question_mask,
          entity_type_dict,
      )
  
      # (batch_size, num_question_tokens, embedding_dim)
      link_embedding = util.weighted_sum(
          entity_embeddings, linking_probabilities
      )
      encoder_input = torch.cat([link_embedding, embedded_question], 2)
  
      # (batch_size, question_length, encoder_output_dim)
      encoder_outputs = self._dropout(
          self._encoder(encoder_input, question_mask)
      )
  
      # This will be our initial hidden state and memory cell for the decoder LSTM.
      final_encoder_output = util.get_final_encoder_states(
          encoder_outputs,
          question_mask,
          self._encoder.is_bidirectional(),
      )
      memory_cell = encoder_outputs.new_zeros(
          batch_size, self._encoder.get_output_dim()
      )
  
      initial_score = embedded_question.data.new_zeros(batch_size)
  
      (
          action_embeddings,
          output_action_embeddings,
          action_biases,
          action_indices,
      ) = self._embed_actions(actions)
  
      _, num_entities, num_question_tokens = linking_scores.size()
      (
          flattened_linking_scores,
          actions_to_entities,
      ) = self._map_entity_productions(linking_scores, world, actions)
      # To make grouping states together in the decoder easier, we convert the batch dimension in
      # all of our tensors into an outer list.  For instance, the encoder outputs have shape
      # `(batch_size, question_length, encoder_output_dim)`.  We need to convert this into a list
      # of `batch_size` tensors, each of shape `(question_length, encoder_output_dim)`.  Then we
      # won't have to do any index selects, or anything, we'll just do some `torch.cat()`s.
      initial_score_list = [initial_score[i] for i in range(batch_size)]
      encoder_output_list = [
          encoder_outputs[i] for i in range(batch_size)
      ]
      question_mask_list = [question_mask[i] for i in range(batch_size)]
      initial_rnn_state = []
      for i in range(batch_size):
          initial_rnn_state.append(
              RnnState(
                  final_encoder_output[i],
                  memory_cell[i],
                  self._first_action_embedding,
                  self._first_attended_question,
                  encoder_output_list,
                  question_mask_list,
              )
          )
      initial_grammar_state = [
          self._create_grammar_state(world[i], actions[i])
          for i in range(batch_size)
      ]
      initial_state_world = (
          world if add_world_to_initial_state else None
      )
      initial_state = WikiTablesDecoderState(
          batch_indices=range(batch_size),
          action_history=[[] for _ in range(batch_size)],
          score=initial_score_list,
          rnn_state=initial_rnn_state,
          grammar_state=initial_grammar_state,
          action_embeddings=action_embeddings,
          output_action_embeddings=output_action_embeddings,
          action_biases=action_biases,
          action_indices=action_indices,
          possible_actions=actions,
          flattened_linking_scores=flattened_linking_scores,
          actions_to_entities=actions_to_entities,
          entity_types=entity_type_dict,
          world=initial_state_world,
          example_lisp_string=example_lisp_string,
          checklist_state=checklist_states,
          debug_info=None,
      )
      return {
          "initial_state": initial_state,
          "linking_scores": linking_scores,
          "feature_scores": feature_scores,
          "similarity_scores": question_entity_similarity_max_score,
      }
  
imports: from __future__ import absolute_import
  import torch
  from allennlp.common.checks import check_dimensions_match
  from allennlp.common.util import pad_sequence_to_length
  from allennlp.data import Vocabulary
  from allennlp.data.fields.production_rule_field import ProductionRuleArray
  from allennlp.models.model import Model
  from allennlp.models.semantic_parsing.wikitables.wikitables_decoder_state import WikiTablesDecoderState
  from allennlp.modules import Embedding, Seq2SeqEncoder, Seq2VecEncoder, TextFieldEmbedder, TimeDistributed
  from allennlp.modules.seq2vec_encoders import BagOfEmbeddingsEncoder
  from allennlp.nn import util
  from allennlp.nn.decoding import GrammarState, RnnState, ChecklistState
  from allennlp.semparse.type_declarations import type_declaration
  from allennlp.semparse.type_declarations.type_declaration import START_SYMBOL
  from allennlp.semparse.worlds import WikiTablesWorld
  from allennlp.training.metrics import Average, WikiTablesAccuracy
label_desc: Use image node_id 2 for calling the WikiTablesSemanticParser obj's underlying member method code with example usage: obj._get_initial_state_and_scores(question, table, world, actions, example_lisp_string, add_world_to_initial_state, checklist_states) and returns: dict