

# Quantized-distilbert-banking77

This model is a statically quantized version of [optimum/distilbert-base-uncased-finetuned-banking77](https://huggingface.co/optimum/distilbert-base-uncased-finetuned-banking77) on the `banking77` dataset.

The model was created using the [optimum-static-quantization](https://github.com/philschmid/optimum-static-quantization) notebook.

It achieves the following results on the evaluation set:

**Accuracy**

- Vanilla model: 92.5%
- Quantized model: 92.24%  

> The quantized model achieves 99.72% accuracy of the fp32 model

**Latency**

Payload sequence length: 128  
Instance type: AWS c6i.xlarge  

| latency | vanilla transformers | quantized optimum model | improvement |
|---------|----------------------|-------------------------|-------------|
| p95     | 75.69ms              | 26.75ms                 | 2.83x       |
| avg     | 57.52ms              | 24.86ms                 | 2.31x       |

## How to use

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained("philschmid/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("philschmid/quantized-distilbert-banking77")

remote_clx = pipeline("text-classification",model=model, tokenizer=tokenizer)

remote_clx("What is the exchange rate like on this app?")
```
{"tags": ["optimum"], "datasets": ["banking77"], "metrics": ["accuracy"], "model-index": [{"name": "quantized-distilbert-banking77", "results": [{"task": {"type": "text-classification", "name": "Text Classification"}, "dataset": {"name": "banking77", "type": "banking77"}, "metrics": [{"type": "accuracy", "value": 0.9224, "name": "Accuracy"}]}]}]}
transformers onnx text-classification optimum dataset:banking77 model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 

Quantized-distilbert-banking77
==============================


This model is a statically quantized version of optimum/distilbert-base-uncased-finetuned-banking77 on the 'banking77' dataset.


The model was created using the optimum-static-quantization notebook.


It achieves the following results on the evaluation set:


Accuracy


* Vanilla model: 92.5%
* Quantized model: 92.24%



> 
> The quantized model achieves 99.72% accuracy of the fp32 model
> 
> 
> 


Latency


Payload sequence length: 128  

Instance type: AWS URL



How to use
----------



TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #onnx #text-classification #optimum #dataset-banking77 #model-index #autotrain_compatible #endpoints_compatible #region-us 
