class_docstr: This wrapper utility function implements a single beam search to generate efficiently text.
  A lot of the credit goes to the huggingface team and its chief scientist Thomas Wolf whose implementation I based
  myself off.
  
  Args:
      encoder: huggingface encoder or onnx session for the encoder of T5. Can be obtained with the
          create_t5_encoder_decoder utility function for pytorch, see examples below.
      decoder_with_lm_head: decoder with language model head on top. Can be obtained with the
          create_t5_encoder_decoder utility function for pytorch, see examples below.
      tokenizer: huggingface tokenizer
      onnx (bool): whether to use onnx or the default pytorch
      cuda (bool): whether to use cuda or the cpu
  
  Examples:
      For pytorch:
      >>> from transformers import T5Tokenizer
      >>> from onnxt5 import create_t5_encoder_decoder, GenerativeT5
      >>> pretrained_model = 't5-base' # This can be a pretrained version, or the path to a huggingface model
      >>> simplified_encoder, decoder_with_lm_head = create_t5_encoder_decoder(pretrained_model)
      >>> tokenizer = T5Tokenizer.from_pretrained(pretrained_model)
      >>> generative_t5 = GenerativeT5(simplified_encoder, decoder_with_lm_head, tokenizer)
      >>> generative_t5('translate English to French: I was a victim of a series of accidents.', 16, temperature=0.)[0]
      >>> # Output: "Je suis victime d'une série d'accidents."
  
      For onnx:
      >>> from transformers import T5Tokenizer
      >>> from onnxruntime import InferenceSession
      >>> from onnxt5 import GenerativeT5
      >>> decoder_sess = InferenceSession('~/t5-decoder-with-lm-head.onnx')
      >>> encoder_sess = InferenceSession('~/t5-encoder.onnx')
      >>> tokenizer = T5Tokenizer.from_pretrained(pretrained_model)
      >>> generative_t5 = GenerativeT5(encoder_sess, decoder_sess, tokenizer, onnx=True)
      >>> generative_t5('translate English to French: I was a victim of a series of accidents.', 16, temperature=0.)[0]
      >>> # Output: "Je suis victime d'une série d'accidents."
docstr: Forward function to generate text after a prompt
  Args:
      prompt: str to run (don't forget to add at the beginning the task to run such as "summarize:"
              or "translate English to German:"
      max_context_length: maximum number of tokens to use as context
docstr_tok: ["Forward","function","to","generate","text","after","a","prompt","Args",":","prompt",":","str","to","run","(","do","n't","forget","to","add","at","the","beginning","the","task","to","run","such","as","``","summarize",":","''","or","``","translate","English","to","German",":","''","max_context_length",":","maximum","number","of","tokens","to","use","as","context"]
code: def forward(
      self,
      prompt,
      max_length,
      temperature=1.0,
      repetition_penalty=1.0,
      top_k=50,
      top_p=0,
      max_context_length=512,
  ):
      """Forward function to generate text after a prompt
      Args:
          prompt: str to run (don't forget to add at the beginning the task to run such as "summarize:"
                  or "translate English to German:"
          max_context_length: maximum number of tokens to use as context
  
      """
      with torch.no_grad():
          new_tokens = torch.tensor(())
          new_logits = []
          generated = torch.tensor(self.tokenizer(prompt)["input_ids"])[
              : max_context_length - 1
          ].unsqueeze(0)
          if self.cuda and not self.onnx:
              generated = generated.cuda()
          temperature = temperature
          # Getting encoder past
          if self.onnx:
              encoder_outputs_prompt = self.encoder.run(
                  None, {"input_ids": generated.cpu().numpy()}
              )[0]
          else:
              encoder_outputs_prompt = self.encoder(generated)
          repetition_penalty = repetition_penalty
          top_k = top_k
          top_p = top_p
  
          # The sequence now needs to start with a
          generated = torch.zeros((1, 1), dtype=torch.long)
          if self.cuda and not self.onnx:
              generated = generated.cuda()
  
          for _ in trange(max_length):
              if self.onnx:
                  outputs = torch.tensor(
                      self.decoder_with_lm_head.run(
                          None,
                          {
                              "input_ids": generated.cpu().numpy(),
                              "encoder_hidden_states": encoder_outputs_prompt,
                          },
                      )[0][0]
                  )
              else:
                  outputs = self.decoder_with_lm_head(
                      input_ids=generated,
                      encoder_hidden_states=encoder_outputs_prompt,
                  )[0]
              next_token_logits = outputs[-1, :] / (
                  temperature if temperature > 0 else 1.0
              )
              if int(next_token_logits.argmax()) == 1:
                  break
              new_logits.append(next_token_logits)
              for _ in set(generated.view(-1).tolist()):
                  next_token_logits[_] /= repetition_penalty
              if temperature == 0:  # greedy sampling:
                  next_token = torch.argmax(
                      next_token_logits
                  ).unsqueeze(0)
              else:
                  filtered_logits = top_k_top_p_filtering(
                      next_token_logits, top_k=top_k, top_p=top_p
                  )
                  next_token = torch.multinomial(
                      F.softmax(filtered_logits, dim=-1), num_samples=1
                  )
              generated = torch.cat(
                  (generated, next_token.unsqueeze(0)), dim=1
              )
              new_tokens = torch.cat((new_tokens, next_token), 0)
  
          return self.tokenizer.decode(new_tokens), new_logits
  
file_path: onnxt5/onnxt5/models.py
imports: import torch
  import torch.nn.functional
  from tqdm import trange
label_desc: Use image node_id 2 for calling the GenerativeT5 obj's underlying member method code with example usage: obj.forward(prompt, max_length, temperature, repetition_penalty, top_k, top_p, max_context_length) and returns: self,  new_logits