class_docstr: An OutputSeq object is the result of running a language model on some input data. It contains not only the output
  sequence of words generated by the model, but also other data collecting during the generation process
  that is useful to analyze the model.
  
  In addition to the data, the object has methods to create plots
  and visualizations of that collected data. These include:
  
  - [layer_predictions()](./#ecco.output.OutputSeq.layer_predictions) <br/>
  Which tokens did the model consider as the best outputs for a specific position in the sequence?
  - [rankings()](./#ecco.output.OutputSeq.rankings) <br/>
  After the model chooses an output token for a specific position, this visual looks back at the ranking
  of this token at each layer of the model when it was generated (layers assign scores to candidate output tokens,
  the higher the "probability" score, the higher the ranking of the token).
  - [rankings_watch()](./#ecco.output.OutputSeq.rankings_watch) <br />
  Shows the rankings of multiple tokens as the model scored them for a single position. For example, if the input is
  "The cat \_\_\_", we use this method to observe how the model ranked the words "is", "are", "was" as candidates
  to fill in the blank.
  - [primary_attributions()](./#ecco.output.OutputSeq.primary_attributions) <br />
  How important was each input token in the selection of calculating the output token?
  
  
  To process neuron activations, OutputSeq has methods to reduce the dimensionality and reveal underlying patterns in
  neuron firings. These are:
  
  - [run_nmf()](./#ecco.output.OutputSeq.run_nmf)
class_docstr_tok: ["An","OutputSeq","object","is","the","result","of","running","a","language","model","on","some","input","data",".","It","contains","not","only","the","output","sequence","of","words","generated","by","the","model",",","but","also","other","data","collecting","during","the","generation","process","that","is","useful","to","analyze","the","model",".","In","addition","to","the","data",",","the","object","has","methods","to","create","plots","and","visualizations","of","that","collected","data",".","These","include",":","-","[","layer_predictions","(",")","]","(",".\/","#","ecco.output.OutputSeq.layer_predictions",")","<","br\/",">","Which","tokens","did","the","model","consider","as","the","best","outputs","for","a","specific","position","in","the","sequence","?","-","[","rankings","(",")","]","(",".\/","#","ecco.output.OutputSeq.rankings",")","<","br\/",">","After","the","model","chooses","an","output","token","for","a","specific","position",",","this","visual","looks","back","at","the","ranking","of","this","token","at","each","layer","of","the","model","when","it","was","generated","(","layers","assign","scores","to","candidate","output","tokens",",","the","higher","the","``","probability","''","score",",","the","higher","the","ranking","of","the","token",")",".","-","[","rankings_watch","(",")","]","(",".\/","#","ecco.output.OutputSeq.rankings_watch",")","<","br","\/",">","Shows","the","rankings","of","multiple","tokens","as","the","model","scored","them","for","a","single","position",".","For","example",",","if","the","input","is","''","The","cat","\\_\\_\\_","''",",","we","use","this","method","to","observe","how","the","model","ranked","the","words","``","is","''",",","``","are","''",",","``","was","''","as","candidates","to","fill","in","the","blank",".","-","[","primary_attributions","(",")","]","(",".\/","#","ecco.output.OutputSeq.primary_attributions",")","<","br","\/",">","How","important","was","each","input","token","in","the","selection","of","calculating","the","output","token","?","To","process","neuron","activations",",","OutputSeq","has","methods","to","reduce","the","dimensionality","and","reveal","underlying","patterns","in","neuron","firings",".","These","are",":","-","[","run_nmf","(",")","]","(",".\/","#","ecco.output.OutputSeq.run_nmf",")"]
docstr:             Explorable showing primary attributions of each token generation step.
              Hovering-over or tapping an output token imposes a saliency map on other tokens
              showing their importance as features to that prediction.
  
              Examples:
  
              ```python
              import ecco
              lm = ecco.from_pretrained('distilgpt2')
              text= "The countries of the European Union are:
  1. Austria
  2. Belgium
  3. Bulgaria
  4."
              output = lm.generate(text, generate=20, do_sample=True)
  
              # Show primary attributions explorable
              output.primary_attributions()
              ```
  
              Which creates the following interactive explorable:
              ![input saliency example 1](../../img/saliency_ex_1.png)
  
              If we want more details on the saliency values, we can use the detailed view:
  
              ```python
              # Show detailed explorable
              output.primary_attributions(style="detailed")
              ```
  
              Which creates the following interactive explorable:
  
              ![input saliency example 2 - detailed](../../img/saliency_ex_2.png)
  
  
              Details:
              This view shows the Gradient * Inputs method of input saliency. The attribution values are calculated across the
              embedding dimensions, then we use the L2 norm to calculate a score for each token (from the values of its embeddings dimension)
              To get a percentage value, we normalize the scores by dividing by the sum of the attribution scores for all
              the tokens in the sequence.
          
docstr_tok: ["Explorable","showing","primary","attributions","of","each","token","generation","step",".","Hovering-over","or","tapping","an","output","token","imposes","a","saliency","map","on","other","tokens","showing","their","importance","as","features","to","that","prediction",".","Examples",":","``","`","python","import","ecco","lm","=","ecco.from_pretrained","(","'distilgpt2","'",")","text=","``","The","countries","of","the","European","Union","are",":","1",".","Austria","2",".","Belgium","3",".","Bulgaria","4",".","''","output","=","lm.generate","(","text",",","generate=20",",","do_sample=True",")","#","Show","primary","attributions","explorable","output.primary_attributions","(",")","``","`","Which","creates","the","following","interactive","explorable",":","!","[","input","saliency","example","1","]","(","..","\/","..","\/img\/saliency_ex_1.png",")","If","we","want","more","details","on","the","saliency","values",",","we","can","use","the","detailed","view",":","``","`","python","#","Show","detailed","explorable","output.primary_attributions","(","style=","''","detailed","''",")","``","`","Which","creates","the","following","interactive","explorable",":","!","[","input","saliency","example","2","-","detailed","]","(","..","\/","..","\/img\/saliency_ex_2.png",")","Details",":","This","view","shows","the","Gradient","*","Inputs","method","of","input","saliency",".","The","attribution","values","are","calculated","across","the","embedding","dimensions",",","then","we","use","the","L2","norm","to","calculate","a","score","for","each","token","(","from","the","values","of","its","embeddings","dimension",")","To","get","a","percentage","value",",","we","normalize","the","scores","by","dividing","by","the","sum","of","the","attribution","scores","for","all","the","tokens","in","the","sequence","."]
code: def primary_attributions(
      self,
      attr_method: Optional[str] = "grad_x_input",
      style="minimal",
      ignore_tokens: Optional[List[int]] = [],
      **kwargs,
  ):
      """
      Explorable showing primary attributions of each token generation step.
      Hovering-over or tapping an output token imposes a saliency map on other tokens
      showing their importance as features to that prediction.
  
      Examples:
  
      ```python
      import ecco
      lm = ecco.from_pretrained('distilgpt2')
      text= "The countries of the European Union are:\n1. Austria\n2. Belgium\n3. Bulgaria\n4."
      output = lm.generate(text, generate=20, do_sample=True)
  
      # Show primary attributions explorable
      output.primary_attributions()
      ```
  
      Which creates the following interactive explorable:
      ![input saliency example 1](../../img/saliency_ex_1.png)
  
      If we want more details on the saliency values, we can use the detailed view:
  
      ```python
      # Show detailed explorable
      output.primary_attributions(style="detailed")
      ```
  
      Which creates the following interactive explorable:
  
      ![input saliency example 2 - detailed](../../img/saliency_ex_2.png)
  
  
      Details:
      This view shows the Gradient * Inputs method of input saliency. The attribution values are calculated across the
      embedding dimensions, then we use the L2 norm to calculate a score for each token (from the values of its embeddings dimension)
      To get a percentage value, we normalize the scores by dividing by the sum of the attribution scores for all
      the tokens in the sequence.
      """
      position = self.n_input_tokens
  
      importance_id = position - self.n_input_tokens
      tokens = []
  
      assert (
          attr_method in self.attribution
      ), f"attr_method={attr_method} not found. Choose one of the following: {list(self.attribution.keys())}"
      attribution = self.attribution[attr_method]
      for idx, token in enumerate(self.tokens[0]):
          token_id = self.token_ids[0][idx]
          raw_token = self.tokenizer.convert_ids_to_tokens([token_id])[
              0
          ]
          clean_token = self.tokenizer.decode(token_id)
          # Strip prefixes because bert decode still has ## for partials even after decode()
          clean_token = strip_tokenizer_prefix(self.config, clean_token)
  
          type = "input" if idx < self.n_input_tokens else "output"
          if idx < len(attribution[importance_id]):
              imp = attribution[importance_id][idx]
          else:
              imp = 0
  
          tokens.append(
              {
                  "token": clean_token,
                  "token_id": int(self.token_ids[0][idx]),
                  "is_partial": is_partial_token(
                      self.config, raw_token
                  ),
                  "type": type,
                  "value": str(
                      imp
                  ),  # because json complains of floats. Probably not used?
                  "position": idx,
              }
          )
  
      if len(ignore_tokens) > 0:
          for output_token_index, _ in enumerate(attribution):
              for idx in ignore_tokens:
                  attribution[output_token_index][idx] = 0
  
      data = {
          "tokens": tokens,
          "attributions": [att.tolist() for att in attribution],
      }
  
      d.display(
          d.HTML(
              filename=os.path.join(self._path, "html", "setup.html")
          )
      )
  
      if style == "minimal":
          js = f"""
               requirejs(['basic', 'ecco'], function(basic, ecco){{
                  const viz_id = basic.init()
                  console.log(viz_id)
                  // ecco.interactiveTokens(viz_id, {{}})
                  window.ecco[viz_id] = new ecco.MinimalHighlighter({{
                      parentDiv: viz_id,
                      data: {json.dumps(data)},
                      preset: 'viridis',
                      tokenization_config: {json.dumps(self.config['tokenizer_config'])}
  
               }})
  
               window.ecco[viz_id].init();
               window.ecco[viz_id].selectFirstToken();
  
               }}, function (err) {{
                  console.log(err);
              }})"""
      elif style == "detailed":
          js = f"""
               requirejs(['basic', 'ecco'], function(basic, ecco){{
                  const viz_id = basic.init()
                  console.log(viz_id)
                  window.ecco[viz_id] = ecco.interactiveTokens({{
                      parentDiv: viz_id,
                      data: {json.dumps(data)},
                      tokenization_config: {json.dumps(self.config['tokenizer_config'])}
               }})
  
               }}, function (err) {{
                  console.log(err);
              }})"""
  
      d.display(d.Javascript(js))
  
      if "printJson" in kwargs and kwargs["printJson"]:
          print(data)
          return data
  
imports: import os
  import json
  import ecco
  from IPython import display
  from ecco import util, lm_plots
  import random
  import matplotlib.pyplot
  import numpy
  import torch
  from torch.nn import functional
  from sklearn import decomposition
  from typing import Dict, Optional, List, Tuple, Union
  from ecco.util import strip_tokenizer_prefix, is_partial_token
label_desc: Use image node_id 9 for calling the OutputSeq obj's underlying member method code with example usage: obj.primary_attributions(attr_method, style, ignore_tokens) and returns: data