class_docstr: `CandidateGenerator` class to be used for assisted generation and speculative decoding. This class generates
  candidates through the use of a smaller model. Read the following blog post for more information:
  https://huggingface.co/blog/assisted-generation
  
  Args:
      input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
          Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
      assistant_model (`PreTrainedModel`):
          The model to be used for generating candidates. This model should be smaller than the main model.
      generation_config (`~generation.GenerationConfig`, *optional*):
          The generation configuration to be used as base parametrization for the generation call.
      logits_processor (`LogitsProcessorList`):
          An instance of [`LogitsProcessorList`]. List of instances of class derived from [`LogitsProcessor`]
          used to modify the prediction scores of the language modeling head applied at each generation step.
      model_kwargs (`Dict`):
          The keyword arguments that will be passed to the main model, and are used as base inputs for the assistant
          model as well.
      inputs_tensor (`torch.Tensor`, *optional*):
          The model input tensor. In encoder-decoder models, this is the encoder input.
class_docstr_tok: ["`","CandidateGenerator","`","class","to","be","used","for","assisted","generation","and","speculative","decoding",".","This","class","generates","candidates","through","the","use","of","a","smaller","model",".","Read","the","following","blog","post","for","more","information",":","https",":","\/\/huggingface.co\/blog\/assisted-generation","Args",":","input_ids","(","`","torch.LongTensor","`","of","shape","`","(","batch_size",",","sequence_length",")","`",")",":","Indices","of","input","sequence","tokens","in","the","vocabulary",".","[","What","are","input","IDs","?","]","(","..","\/glossary","#","input-ids",")","assistant_model","(","`","PreTrainedModel","`",")",":","The","model","to","be","used","for","generating","candidates",".","This","model","should","be","smaller","than","the","main","model",".","generation_config","(","`","~generation.GenerationConfig","`",",","*","optional","*",")",":","The","generation","configuration","to","be","used","as","base","parametrization","for","the","generation","call",".","logits_processor","(","`","LogitsProcessorList","`",")",":","An","instance","of","[","`","LogitsProcessorList","`","]",".","List","of","instances","of","class","derived","from","[","`","LogitsProcessor","`","]","used","to","modify","the","prediction","scores","of","the","language","modeling","head","applied","at","each","generation","step",".","model_kwargs","(","`","Dict","`",")",":","The","keyword","arguments","that","will","be","passed","to","the","main","model",",","and","are","used","as","base","inputs","for","the","assistant","model","as","well",".","inputs_tensor","(","`","torch.Tensor","`",",","*","optional","*",")",":","The","model","input","tensor",".","In","encoder-decoder","models",",","this","is","the","encoder","input","."]
docstr: Updates the candidate generation strategy based on the outcomes.
  
  Args:
      input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
          Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
      scores (`torch.FloatTensor` of shape `(batch_size, candidate_length, config.vocab_size)`):
          Prediction scores of a language modeling head. These can be logits for each vocabulary when not using
          beam search or log softmax for each vocabulary token when using beam search
      num_matches (`int`):
          The number of matches between the candidate sequences and the model predictions.
docstr_tok: ["Updates","the","candidate","generation","strategy","based","on","the","outcomes",".","Args",":","input_ids","(","`","torch.LongTensor","`","of","shape","`","(","batch_size",",","sequence_length",")","`",")",":","Indices","of","input","sequence","tokens","in","the","vocabulary",".","[","What","are","input","IDs","?","]","(","..","\/glossary","#","input-ids",")","scores","(","`","torch.FloatTensor","`","of","shape","`","(","batch_size",",","candidate_length",",","config.vocab_size",")","`",")",":","Prediction","scores","of","a","language","modeling","head",".","These","can","be","logits","for","each","vocabulary","when","not","using","beam","search","or","log","softmax","for","each","vocabulary","token","when","using","beam","search","num_matches","(","`","int","`",")",":","The","number","of","matches","between","the","candidate","sequences","and","the","model","predictions","."]
code: def update_candidate_strategy(
      self,
      input_ids: torch.LongTensor,
      scores: torch.FloatTensor,
      num_matches: int,
  ):
      """
      Updates the candidate generation strategy based on the outcomes.
  
      Args:
          input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
              Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)
          scores (`torch.FloatTensor` of shape `(batch_size, candidate_length, config.vocab_size)`):
              Prediction scores of a language modeling head. These can be logits for each vocabulary when not using
              beam search or log softmax for each vocabulary token when using beam search
          num_matches (`int`):
              The number of matches between the candidate sequences and the model predictions.
      """
      # Adjust the max number of assistant tokens to use in the next iteration. This is a simple heuristic,
      # probably can be improved -- we want to balance the benefits of getting assistant tokens correct with the
      # cost of forecasting incorrect assistant tokens.
      if (
          self.assistant_model.generation_config.num_assistant_tokens_schedule
          == "heuristic"
      ):
          if num_matches == int(self.num_assistant_tokens):
              self.num_assistant_tokens += 2.0
          else:
              self.num_assistant_tokens = max(
                  1.0, self.num_assistant_tokens - 1.0
              )
  
file_path: transformers/src/transformers/generation/candidate_generator.py
imports: import copy
  from typing import TYPE_CHECKING, Any, Dict, Optional, Tuple
  import torch
label_desc: Use image node_id 3 for calling the AssistedCandidateGenerator obj's underlying member method code with example usage: obj.update_candidate_strategy(input_ids, scores, num_matches) without return types