class_docstr: Wrapper for a Huggingface Transformers GPT-2 model.
  
  This class loads a tokenizer and model using the Huggingface library and
  provides the LIT-required functions plus additional helper functions to
  convert and clean tokens and to compute the top_k predictions from logits.
class_docstr_tok: ["Wrapper","for","a","Huggingface","Transformers","GPT-2","model",".","This","class","loads","a","tokenizer","and","model","using","the","Huggingface","library","and","provides","the","LIT-required","functions","plus","additional","helper","functions","to","convert","and","clean","tokens","and","to","compute","the","top_k","predictions","from","logits","."]
docstr: Predicts one batch of tokenized text.
  
  Also performs some batch-level post-processing in TF.
  Single-example postprocessing is done in _postprocess(), and operates on
  numpy arrays.
  
  Each prediction has the following returns:
  logits: tf.Tensor (batch_size, sequence_length, config.vocab_size).
  past: list[tf.Tensor] of length config.n_layers with each tensor shape
           (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
  states: Tuple of tf.Tensor (one for embeddings + one for each layer),
          with shape (batch_size, sequence_length, hidden_size).
  attentions: Tuple of tf.Tensor (one for each layer) with shape
              (batch_size, num_heads, sequence_length, sequence_length)
  Within this function, we combine each Tuple/List into a single Tensor.
  
  Args:
    encoded_inputs: output of self.tokenizer.batch_encode_plus()
  
  Returns:
    payload: Dictionary with items described above, each as single Tensor.
docstr_tok: ["Predicts","one","batch","of","tokenized","text",".","Also","performs","some","batch-level","post-processing","in","TF",".","Single-example","postprocessing","is","done","in","_postprocess","(",")",",","and","operates","on","numpy","arrays",".","Each","prediction","has","the","following","returns",":","logits",":","tf.Tensor","(","batch_size",",","sequence_length",",","config.vocab_size",")",".","past",":","list","[","tf.Tensor","]","of","length","config.n_layers","with","each","tensor","shape","(","2",",","batch_size",",","num_heads",",","sequence_length",",","embed_size_per_head",")",")",".","states",":","Tuple","of","tf.Tensor","(","one","for","embeddings","+","one","for","each","layer",")",",","with","shape","(","batch_size",",","sequence_length",",","hidden_size",")",".","attentions",":","Tuple","of","tf.Tensor","(","one","for","each","layer",")","with","shape","(","batch_size",",","num_heads",",","sequence_length",",","sequence_length",")","Within","this","function",",","we","combine","each","Tuple\/List","into","a","single","Tensor",".","Args",":","encoded_inputs",":","output","of","self.tokenizer.batch_encode_plus","(",")","Returns",":","payload",":","Dictionary","with","items","described","above",",","each","as","single","Tensor","."]
code: def _pred(self, encoded_inputs):
      """Predicts one batch of tokenized text.
  
      Also performs some batch-level post-processing in TF.
      Single-example postprocessing is done in _postprocess(), and operates on
      numpy arrays.
  
      Each prediction has the following returns:
      logits: tf.Tensor (batch_size, sequence_length, config.vocab_size).
      past: list[tf.Tensor] of length config.n_layers with each tensor shape
               (2, batch_size, num_heads, sequence_length, embed_size_per_head)).
      states: Tuple of tf.Tensor (one for embeddings + one for each layer),
              with shape (batch_size, sequence_length, hidden_size).
      attentions: Tuple of tf.Tensor (one for each layer) with shape
                  (batch_size, num_heads, sequence_length, sequence_length)
      Within this function, we combine each Tuple/List into a single Tensor.
  
      Args:
        encoded_inputs: output of self.tokenizer.batch_encode_plus()
  
      Returns:
        payload: Dictionary with items described above, each as single Tensor.
      """
      out: transformers.modeling_tf_outputs.TFCausalLMOutputWithPast = (
          self.model(encoded_inputs["input_ids"])
      )
  
      model_probs = tf.nn.softmax(out.logits, axis=-1)
      top_k = tf.math.top_k(
          model_probs, k=self.top_k, sorted=True, name=None
      )
      batched_outputs = {
          "input_ids": encoded_inputs["input_ids"],
          "ntok": tf.reduce_sum(
              encoded_inputs["attention_mask"], axis=1
          ),
          "top_k_indices": top_k.indices,
          "top_k_probs": top_k.values,
      }
  
      # Convert representations for each layer from tuples to single Tensor.
      for i in range(len(out.attentions)):
          batched_outputs[f"layer_{i+1:d}_attention"] = out.attentions[
              i
          ]
      for i in range(len(out.hidden_states)):
          batched_outputs[
              f"layer_{i:d}_avg_embedding"
          ] = tf.math.reduce_mean(out.hidden_states[i], axis=1)
  
      return batched_outputs
  
imports: import re
  from lit_nlp.api import model
  from lit_nlp.api import types
  from lit_nlp.examples.models import model_utils
  from lit_nlp.lib import file_cache
  from lit_nlp.lib import utils
  import numpy
  import tensorflow
  import transformers
label_desc: Use image node_id 6 for calling the GPT2LanguageModel obj's underlying member method code with example usage: obj._pred(encoded_inputs) and returns: batched_outputs