wzuidema
commited on
added some explanation
Browse files
app.py
CHANGED
@@ -282,14 +282,17 @@ lig = gradio.Interface(
|
|
282 |
iface = gradio.Parallel(hila, lig,
|
283 |
title="RoBERTa Explainability",
|
284 |
description="""
|
285 |
-
|
|
|
|
|
286 |
|
287 |
-
|
288 |
-
|
289 |
-
|
|
|
|
|
290 |
[Transformer-MM_explainability](https://github.com/hila-chefer/Transformer-MM-Explainability/))
|
291 |
-
|
292 |
-
* [captum](https://captum.ai/)'s LayerIntegratedGradients
|
293 |
""",
|
294 |
examples=[
|
295 |
[
|
|
|
282 |
iface = gradio.Parallel(hila, lig,
|
283 |
title="RoBERTa Explainability",
|
284 |
description="""
|
285 |
+
In this demo, we use the RoBERTa language model (optimized for masked language modelling and finetuned for sentiment analysis).
|
286 |
+
The model predicts for a given sentences whether it expresses a positive, negative or neutral sentiment.
|
287 |
+
But how does it arrive at its classification? A range of so-called "attribution methods" have been developed that attempt to determine the importance of the words in the input for the final prediction.
|
288 |
|
289 |
+
(Note that in general, importance scores only provide a very limited form of "explanation" and that different attribution methods differ radically in how they assign importance).
|
290 |
+
|
291 |
+
Two key methods for Transformers are "attention rollout" (Abnar & Zuidema, 2020) and (layered) Integrated Gradient. Here we show:
|
292 |
+
|
293 |
+
* Gradient-weighted attention rollout, as defined by [Hila Chefer's](https://github.com/hila-chefer)
|
294 |
[Transformer-MM_explainability](https://github.com/hila-chefer/Transformer-MM-Explainability/))
|
295 |
+
* Layer IG, as implemented in [Captum](https://captum.ai/)'s LayerIntegratedGradients
|
|
|
296 |
""",
|
297 |
examples=[
|
298 |
[
|