class_docstr: ``NlvrSemanticCoverageParser`` is an ``NlvrSemanticParser`` that gets around the problem of lack
  of annotated logical forms by maximizing coverage of the output sequences over a prespecified
  agenda. In addition to the signal from coverage, we also compute the denotations given by the
  logical forms and define a hybrid cost based on coverage and denotation errors. The training
  process then minimizes the expected value of this cost over an approximate set of logical forms
  produced by the parser, obtained by performing beam search.
  
  Parameters
  ----------
  vocab : ``Vocabulary``
      Passed to super-class.
  sentence_embedder : ``TextFieldEmbedder``
      Passed to super-class.
  action_embedding_dim : ``int``
      Passed to super-class.
  encoder : ``Seq2SeqEncoder``
      Passed to super-class.
  attention : ``Attention``
      We compute an attention over the input question at each step of the decoder, using the
      decoder hidden state as the query.  Passed to the DecoderStep.
  beam_size : ``int``
      Beam size for the beam search used during training.
  max_num_finished_states : ``int``, optional (default=None)
      Maximum number of finished states the trainer should compute costs for.
  normalize_beam_score_by_length : ``bool``, optional (default=False)
      Should the log probabilities be normalized by length before renormalizing them? Edunov et
      al. do this in their work, but we found that not doing it works better. It's possible they
      did this because their task is NMT, and longer decoded sequences are not necessarily worse,
      and shouldn't be penalized, while we will mostly want to penalize longer logical forms.
  max_decoding_steps : ``int``
      Maximum number of steps for the beam search during training.
  dropout : ``float``, optional (default=0.0)
      Probability of dropout to apply on encoder outputs, decoder outputs and predicted actions.
  checklist_cost_weight : ``float``, optional (default=0.6)
      Mixture weight (0-1) for combining coverage cost and denotation cost. As this increases, we
      weigh the coverage cost higher, with a value of 1.0 meaning that we do not care about
      denotation accuracy.
  dynamic_cost_weight : ``Dict[str, Union[int, float]]``, optional (default=None)
      A dict containing keys ``wait_num_epochs`` and ``rate`` indicating the number of steps
      after which we should start decreasing the weight on checklist cost in favor of denotation
      cost, and the rate at which we should do it. We will decrease the weight in the following
      way - ``checklist_cost_weight = checklist_cost_weight - rate * checklist_cost_weight``
      starting at the apropriate epoch.  The weight will remain constant if this is not provided.
  penalize_non_agenda_actions : ``bool``, optional (default=False)
      Should we penalize the model for producing terminal actions that are outside the agenda?
  initial_mml_model_file : ``str`` , optional (default=None)
      If you want to initialize this model using weights from another model trained using MML,
      pass the path to the ``model.tar.gz`` file of that model here.
class_docstr_tok: ["``","NlvrSemanticCoverageParser","``","is","an","``","NlvrSemanticParser","``","that","gets","around","the","problem","of","lack","of","annotated","logical","forms","by","maximizing","coverage","of","the","output","sequences","over","a","prespecified","agenda",".","In","addition","to","the","signal","from","coverage",",","we","also","compute","the","denotations","given","by","the","logical","forms","and","define","a","hybrid","cost","based","on","coverage","and","denotation","errors",".","The","training","process","then","minimizes","the","expected","value","of","this","cost","over","an","approximate","set","of","logical","forms","produced","by","the","parser",",","obtained","by","performing","beam","search",".","Parameters","--","--","--","--","--","vocab",":","``","Vocabulary","``","Passed","to","super-class",".","sentence_embedder",":","``","TextFieldEmbedder","``","Passed","to","super-class",".","action_embedding_dim",":","``","int","``","Passed","to","super-class",".","encoder",":","``","Seq2SeqEncoder","``","Passed","to","super-class",".","attention",":","``","Attention","``","We","compute","an","attention","over","the","input","question","at","each","step","of","the","decoder",",","using","the","decoder","hidden","state","as","the","query",".","Passed","to","the","DecoderStep",".","beam_size",":","``","int","``","Beam","size","for","the","beam","search","used","during","training",".","max_num_finished_states",":","``","int","``",",","optional","(","default=None",")","Maximum","number","of","finished","states","the","trainer","should","compute","costs","for",".","normalize_beam_score_by_length",":","``","bool","``",",","optional","(","default=False",")","Should","the","log","probabilities","be","normalized","by","length","before","renormalizing","them","?","Edunov","et","al",".","do","this","in","their","work",",","but","we","found","that","not","doing","it","works","better",".","It","'s","possible","they","did","this","because","their","task","is","NMT",",","and","longer","decoded","sequences","are","not","necessarily","worse",",","and","should","n't","be","penalized",",","while","we","will","mostly","want","to","penalize","longer","logical","forms",".","max_decoding_steps",":","``","int","``","Maximum","number","of","steps","for","the","beam","search","during","training",".","dropout",":","``","float","``",",","optional","(","default=0.0",")","Probability","of","dropout","to","apply","on","encoder","outputs",",","decoder","outputs","and","predicted","actions",".","checklist_cost_weight",":","``","float","``",",","optional","(","default=0.6",")","Mixture","weight","(","0-1",")","for","combining","coverage","cost","and","denotation","cost",".","As","this","increases",",","we","weigh","the","coverage","cost","higher",",","with","a","value","of","1.0","meaning","that","we","do","not","care","about","denotation","accuracy",".","dynamic_cost_weight",":","``","Dict","[","str",",","Union","[","int",",","float","]","]","``",",","optional","(","default=None",")","A","dict","containing","keys","``","wait_num_epochs","``","and","``","rate","``","indicating","the","number","of","steps","after","which","we","should","start","decreasing","the","weight","on","checklist","cost","in","favor","of","denotation","cost",",","and","the","rate","at","which","we","should","do","it",".","We","will","decrease","the","weight","in","the","following","way","-","``","checklist_cost_weight","=","checklist_cost_weight","-","rate","*","checklist_cost_weight","``","starting","at","the","apropriate","epoch",".","The","weight","will","remain","constant","if","this","is","not","provided",".","penalize_non_agenda_actions",":","``","bool","``",",","optional","(","default=False",")","Should","we","penalize","the","model","for","producing","terminal","actions","that","are","outside","the","agenda","?","initial_mml_model_file",":","``","str","``",",","optional","(","default=None",")","If","you","want","to","initialize","this","model","using","weights","from","another","model","trained","using","MML",",","pass","the","path","to","the","``","model.tar.gz","``","file","of","that","model","here","."]
code: def _update_metrics(
      self,
      action_strings,
      worlds,
      label_strings,
      possible_actions,
      agenda_data,
  ):
      # TODO(pradeep): Move this to the base class.
      # TODO(pradeep): action_strings contains k-best lists. This method only uses the top decoded
      # sequence currently. Maybe define top-k metrics?
      batch_size = len(worlds)
      for i in range(batch_size):
          # Using only the top decoded sequence per instance.
          instance_action_strings = (
              action_strings[i][0] if action_strings[i] else []
          )
          sequence_is_correct = [False]
          in_agenda_ratio = 0.0
          instance_possible_actions = possible_actions[i]
          if instance_action_strings:
              terminal_agenda_actions = []
              for rule_id in agenda_data[i]:
                  if rule_id == -1:
                      continue
                  action_string = instance_possible_actions[rule_id][0]
                  right_side = action_string.split(" -> ")[1]
                  if right_side.isdigit() or (
                      "[" not in right_side and len(right_side) > 1
                  ):
                      terminal_agenda_actions.append(action_string)
              actions_in_agenda = [
                  action in instance_action_strings
                  for action in terminal_agenda_actions
              ]
              in_agenda_ratio = sum(actions_in_agenda) / len(
                  actions_in_agenda
              )
              instance_label_strings = label_strings[i]
              instance_worlds = worlds[i]
              sequence_is_correct = self._check_denotation(
                  instance_action_strings,
                  instance_label_strings,
                  instance_worlds,
              )
          for correct_in_world in sequence_is_correct:
              self._denotation_accuracy(1 if correct_in_world else 0)
          self._consistency(1 if all(sequence_is_correct) else 0)
          self._agenda_coverage(in_agenda_ratio)
  
imports: from __future__ import division
  from __future__ import absolute_import
  import logging
  import os
  import torch
  from allennlp.data.fields.production_rule_field import ProductionRuleArray
  from allennlp.data.vocabulary import Vocabulary
  from allennlp.modules import Attention, TextFieldEmbedder, Seq2SeqEncoder
  from allennlp.nn.decoding import DecoderTrainer, ChecklistState
  from allennlp.nn.decoding.decoder_trainers import ExpectedRiskMinimization
  from allennlp.models.archival import load_archive, Archive
  from allennlp.models.model import Model
  from allennlp.models.semantic_parsing.nlvr.nlvr_decoder_state import NlvrDecoderState
  from allennlp.models.semantic_parsing.nlvr.nlvr_decoder_step import NlvrDecoderStep
  from allennlp.models.semantic_parsing.nlvr.nlvr_semantic_parser import NlvrSemanticParser
  from allennlp.semparse.worlds import NlvrWorld
  from allennlp.training.metrics import Average
label_desc: Use image node_id 6 for calling the NlvrCoverageSemanticParser obj's underlying member method code with example usage: obj._update_metrics(action_strings, worlds, label_strings, possible_actions, agenda_data) without return types