

# Quantized-distilbert-banking77

This model is a dynamically quantized version of [optimum/distilbert-base-uncased-finetuned-banking77](https://huggingface.co/optimum/distilbert-base-uncased-finetuned-banking77) on the `banking77` dataset.

The model was created using the [dynamic-quantization](https://github.com/huggingface/workshops/tree/main/mlops-world) notebook from a workshop presented at MLOps World 2022.

It achieves the following results on the evaluation set:

**Accuracy**

- Vanilla model: 92.5%
- Quantized model: 92.44%  

> The quantized model achieves 99.93% accuracy of the FP32 model

**Latency**

Payload sequence length: 128  
Instance type: AWS c6i.xlarge  

| latency | vanilla transformers | quantized optimum model | improvement |
|---------|----------------------|-------------------------|-------------|
| p95     | 63.24ms              | 37.06ms                 | 1.71x       |
| avg     | 62.87ms              | 37.93ms                 | 1.66x       |

## How to use

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained("lewtun/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("lewtun/quantized-distilbert-banking77")

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
classifier("What is the exchange rate like on this app?")
```
{"tags": ["optimum"], "datasets": ["banking77"], "metrics": ["accuracy"], "model-index": [{"name": "quantized-distilbert-banking77", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "banking77", "type": "banking77"}, "metrics": [{"type": "accuracy", "value": 0.9244, "name": "Accuracy"}]}]}]}
transformers onnx text-classification optimum dataset:banking77 model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 

Quantized-distilbert-banking77
==============================


This model is a dynamically quantized version of optimum/distilbert-base-uncased-finetuned-banking77 on the 'banking77' dataset.


The model was created using the dynamic-quantization notebook from a workshop presented at MLOps World 2022.


It achieves the following results on the evaluation set:


Accuracy


* Vanilla model: 92.5%
* Quantized model: 92.44%



> 
> The quantized model achieves 99.93% accuracy of the FP32 model
> 
> 
> 


Latency


Payload sequence length: 128  

Instance type: AWS URL



How to use
----------



TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 
