title: How to calculate the score[logits] of sequences in BeamSearchEncoderDecoderOutput
body: <p>*What are scores in BeamSearchEncoderDecoderOutput.</p>
  <p>*How to get sequences (indices) from <code>scores</code> using simply <code>argmax()</code> ?</p>
  <p>*Why when batch_size &gt;1, why the scores shape is not <strong>(batch_size, beam_nums, num_seq_return, vocab_len)</strong> instead of <strong>(batch_size x beam_nums x num_seq_return, vocab_len)</strong></p>
  <pre><code>from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
  tokenizer = AutoTokenizer.from_pretrained(&quot;t5-small&quot;)
  model = AutoModelForSeq2SeqLM.from_pretrained(&quot;t5-small&quot;)
  model.resize_token_embeddings(len(tokenizer))
  model.to(&quot;cuda&quot;)
  
  # sequences
  seq1 = &quot;summarize: I am confused! I am confused&quot;
  seq2 = &quot;summarize: why generate does not work with batch_size &gt;1&quot;
  
  # encoding input and attention mask
  encoding = tokenizer(
      [seq1, seq2],
      padding=&quot;longest&quot;,
      max_length=128,
      truncation=True,
      return_tensors=&quot;pt&quot;,
  )
  
  input_ids, attention_mask = encoding.input_ids.to(&quot;cuda&quot;), encoding.attention_mask.to(&quot;cuda&quot;)
  output = model.generate(input_ids,
                       max_length=64,
                       early_stopping=True,
                       num_beams=4,
                       do_sample=False,
                       output_scores=True,
                       no_repeat_ngram_size=4,
                       return_dict_in_generate=True,
                       num_return_sequences=1)
  output.sequences
  tokenizer.batch_decode(output.sequences, skip_special_tokens=True)
  
  # output.sequences
  output.sequences
  # tensor([[    0,    27,   183, 11319,    55,     1,     0,     0,     0,     0,
  #              0,     0,     0,     0],
  #         [    0,  3806,   405,    59,   161,    28, 11587,   834,  7991,  2490,
  #            536,     3,     5,     1]], device='cuda:0')
  
  # How to get the above indices using output.scores ??
  # when batch_size = 1, indices seems the same
  </code></pre>