class_docstr: An OutputSeq object is the result of running a language model on some input data. It contains not only the output
  sequence of words generated by the model, but also other data collecting during the generation process
  that is useful to analyze the model.
  
  In addition to the data, the object has methods to create plots
  and visualizations of that collected data. These include:
  
  - [layer_predictions()](./#ecco.output.OutputSeq.layer_predictions) <br/>
  Which tokens did the model consider as the best outputs for a specific position in the sequence?
  - [rankings()](./#ecco.output.OutputSeq.rankings) <br/>
  After the model chooses an output token for a specific position, this visual looks back at the ranking
  of this token at each layer of the model when it was generated (layers assign scores to candidate output tokens,
  the higher the "probability" score, the higher the ranking of the token).
  - [rankings_watch()](./#ecco.output.OutputSeq.rankings_watch) <br />
  Shows the rankings of multiple tokens as the model scored them for a single position. For example, if the input is
  "The cat \_\_\_", we use this method to observe how the model ranked the words "is", "are", "was" as candidates
  to fill in the blank.
  - [primary_attributions()](./#ecco.output.OutputSeq.primary_attributions) <br />
  How important was each input token in the selection of calculating the output token?
  
  
  To process neuron activations, OutputSeq has methods to reduce the dimensionality and reveal underlying patterns in
  neuron firings. These are:
  
  - [run_nmf()](./#ecco.output.OutputSeq.run_nmf)
class_docstr_tok: ["An","OutputSeq","object","is","the","result","of","running","a","language","model","on","some","input","data",".","It","contains","not","only","the","output","sequence","of","words","generated","by","the","model",",","but","also","other","data","collecting","during","the","generation","process","that","is","useful","to","analyze","the","model",".","In","addition","to","the","data",",","the","object","has","methods","to","create","plots","and","visualizations","of","that","collected","data",".","These","include",":","-","[","layer_predictions","(",")","]","(",".\/","#","ecco.output.OutputSeq.layer_predictions",")","<","br\/",">","Which","tokens","did","the","model","consider","as","the","best","outputs","for","a","specific","position","in","the","sequence","?","-","[","rankings","(",")","]","(",".\/","#","ecco.output.OutputSeq.rankings",")","<","br\/",">","After","the","model","chooses","an","output","token","for","a","specific","position",",","this","visual","looks","back","at","the","ranking","of","this","token","at","each","layer","of","the","model","when","it","was","generated","(","layers","assign","scores","to","candidate","output","tokens",",","the","higher","the","``","probability","''","score",",","the","higher","the","ranking","of","the","token",")",".","-","[","rankings_watch","(",")","]","(",".\/","#","ecco.output.OutputSeq.rankings_watch",")","<","br","\/",">","Shows","the","rankings","of","multiple","tokens","as","the","model","scored","them","for","a","single","position",".","For","example",",","if","the","input","is","''","The","cat","\\_\\_\\_","''",",","we","use","this","method","to","observe","how","the","model","ranked","the","words","``","is","''",",","``","are","''",",","``","was","''","as","candidates","to","fill","in","the","blank",".","-","[","primary_attributions","(",")","]","(",".\/","#","ecco.output.OutputSeq.primary_attributions",")","<","br","\/",">","How","important","was","each","input","token","in","the","selection","of","calculating","the","output","token","?","To","process","neuron","activations",",","OutputSeq","has","methods","to","reduce","the","dimensionality","and","reveal","underlying","patterns","in","neuron","firings",".","These","are",":","-","[","run_nmf","(",")","]","(",".\/","#","ecco.output.OutputSeq.run_nmf",")"]
code: def attention(self, attention_values=None, layer=0, **kwargs):
      position = self.n_input_tokens
      # importance_id = position - self.n_input_tokens
  
      importance_id = (
          self.n_input_tokens - 1
      )  # Sete first values to first output token
      tokens = []
      if attention_values:
          attn = attention_values
      else:
          attn = self.attention_values[layer]
          # normalize attention heads
          attn = attn.sum(axis=1) / attn.shape[1]
  
      for idx, token in enumerate(self.tokens):
          # print(idx, attn.shape)
          type = "input" if idx < self.n_input_tokens else "output"
          if idx < len(attn[0][importance_id]):
              attention_value = (
                  attn[0][importance_id][idx].cpu().detach().numpy()
              )
          else:
              attention_value = 0
  
          tokens.append(
              {
                  "token": token,
                  "token_id": int(self.token_ids[idx]),
                  "type": type,
                  "value": str(
                      attention_value
                  ),  # because json complains of floats
                  "position": idx,
              }
          )
  
      data = {
          "tokens": tokens,
          "attributions": [
              att.tolist() for att in attn[0].cpu().detach().numpy()
          ],
      }
  
      d.display(
          d.HTML(
              filename=os.path.join(self._path, "html", "setup.html")
          )
      )
      # d.display(d.HTML(filename=os.path.join(self._path, "html", "basic.html")))
      viz_id = "viz_{}".format(round(random.random() * 1000000))
      js = """
           requirejs(['basic', 'ecco'], function(basic, ecco){{
              const viz_id = basic.init()
  
              ecco.interactiveTokens(viz_id, {})
           }}, function (err) {{
              console.log(err);
          }})""".format(
          data
      )
      d.display(d.Javascript(js))
  
      if "printJson" in kwargs and kwargs["printJson"]:
          print(data)
  
imports: import os
  import json
  import ecco
  from IPython import display
  from ecco import util, lm_plots
  import random
  import matplotlib.pyplot
  import numpy
  import torch
  from torch.nn import functional
  from sklearn import decomposition
  from typing import Dict, Optional, List, Tuple, Union
  from ecco.util import strip_tokenizer_prefix, is_partial_token
label_desc: Use image node_id 14 for calling the OutputSeq obj's underlying member method code with example usage: obj.attention(attention_values, layer) without return types