How to get SHAP values for Huggingface Transformer Model Prediction [Zero-Shot Classification]?
Given a Zero-Shot Classification Task via Huggingface as follows:
from transformers import pipeline
classifier = pipeline(&quot;zero-shot-classification&quot;, model=&quot;facebook/bart-large-mnli&quot;)

example_text = &quot;This is an example text about snowflakes in the summer&quot;
labels = [&quot;weather&quot;, &quot;sports&quot;, &quot;computer industry&quot;]
        
output = classifier(example_text, labels, multi_label=True)
output 
{'sequence': 'This is an example text about snowflakes in the summer',
'labels': ['weather', 'sports'],
'scores': [0.9780895709991455, 0.021910419687628746]}

I am trying to extract the SHAP values to generate a text-based explanation for the prediction result like shown here: SHAP for Transformers
I already tried the following based on the above url:
from transformers import AutoModelForSequenceClassification, AutoTokenizer, ZeroShotClassificationPipeline

model = AutoModelForSequenceClassification.from_pretrained('facebook/bart-large-mnli')
tokenizer = AutoTokenizer.from_pretrained('facebook/bart-large-mnli')

pipe = ZeroShotClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)

def score_and_visualize(text):
    prediction = pipe([text])
    print(prediction[0])

    explainer = shap.Explainer(pipe)
    shap_values = explainer([text])

    shap.plots.text(shap_values)

score_and_visualize(example_text)

Any suggestions? Thanks for your help in advance!
Alternatively to the above pipeline the following also works:
from transformers import AutoModelForSequenceClassification, AutoTokenizer, ZeroShotClassificationPipeline

model = AutoModelForSequenceClassification.from_pretrained('facebook/bart-large-mnli')
tokenizer = AutoTokenizer.from_pretrained('facebook/bart-large-mnli')

classifier = ZeroShotClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)

example_text = &quot;This is an example text about snowflakes in the summer&quot;
labels = [&quot;weather&quot;, &quot;sports&quot;]

output = classifier(example_text, labels)
output 
{'sequence': 'This is an example text about snowflakes in the summer',
'labels': ['weather', 'sports'],
'scores': [0.9780895709991455, 0.021910419687628746]}


The ZeroShotClassificationPipeline is currently not supported by shap, but you can use a workaround. The workaround is required because:

The shap Explainer forwards only one parameter to the model (a pipeline in this case), but the ZeroShotClassificationPipeline requires two parameters, namely text, and labels.
The shap Explainer will access the config of your model and use its label2id and id2label properties. They do not match the labels returned from the ZeroShotClassificationPipeline and will result in an error.

Below is a suggestion for one possible workaround. I recommend opening an issue at shap and requesting official support for huggingface's ZeroShotClassificationPipeline.
import shap
from transformers import AutoModelForSequenceClassification, AutoTokenizer, ZeroShotClassificationPipeline
from typing import Union, List

weights = &quot;valhalla/distilbart-mnli-12-3&quot;

model = AutoModelForSequenceClassification.from_pretrained(weights)
tokenizer = AutoTokenizer.from_pretrained(weights)

# Create your own pipeline that only requires the text parameter 
# for the __call__ method and provides a method to set the labels
class MyZeroShotClassificationPipeline(ZeroShotClassificationPipeline):
    # Overwrite the __call__ method
    def __call__(self, *args):
      o = super().__call__(args[0], self.workaround_labels)[0]

      return [[{&quot;label&quot;:x[0], &quot;score&quot;: x[1]}  for x in zip(o[&quot;labels&quot;], o[&quot;scores&quot;])]]

    def set_labels_workaround(self, labels: Union[str,List[str]]):
      self.workaround_labels = labels

example_text = &quot;This is an example text about snowflakes in the summer&quot;
labels = [&quot;weather&quot;,&quot;sports&quot;]

# In the following, we address issue 2.
model.config.label2id.update({v:k for k,v in enumerate(labels)})
model.config.id2label.update({k:v for k,v in enumerate(labels)})

pipe = MyZeroShotClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)
pipe.set_labels_workaround(labels)

def score_and_visualize(text):
    prediction = pipe([text])
    print(prediction[0])

    explainer = shap.Explainer(pipe)
    shap_values = explainer([text])

    shap.plots.text(shap_values)


score_and_visualize(example_text)

Output:

