
## bert-base-uncased model fine-tuned on QQP

This model was created using the [nn_pruning](https://github.com/huggingface/nn_pruning) python library: the linear layers contains **36%** of the original  weights.


The model contains **50%** of the original weights **overall** (the embeddings account for a significant part of the model, and they are not pruned by this method).


<div class="graph"><script src="/echarlaix/bert-base-uncased-qqp-f87.8-d36-hybrid/raw/main/model_card/density_info.js" id="70162e64-2a82-4147-ac7a-864cfe18a013"></script></div>


## Fine-Pruning details
This model was fine-tuned from the HuggingFace [model](https://huggingface.co/bert-base-uncased) checkpoint on task, and distilled from the model [textattack/bert-base-uncased-QQP](https://huggingface.co/textattack/bert-base-uncased-QQP).
This model is case-insensitive: it does not make a difference between english and English.

A side-effect of block pruning is that some of the attention heads are completely removed: 54 heads were removed on a total of 144 (37.5%).

<div class="graph"><script src="/echarlaix/bert-base-uncased-qqp-f87.8-d36-hybrid/raw/main/model_card/pruning_info.js" id="f4fb8229-3e66-406e-b99f-f771ce6117c8"></script></div>

## Details of the QQP dataset

| Dataset  | Split | # samples |
| -------- | ----- | --------- |
|    QQP   | train | 364K      |
|    QQP   | eval  |  40K      |

### Results

**Pytorch model file size**: `377MB` (original BERT: `420MB`)

| Metric | # Value   | 
| ------ | --------- | 
| **F1** | **87.87** | 




{"language": "en", "license": "apache-2.0", "tags": ["text-classification"], "datasets": ["qqp"], "metrics": ["F1"]}
transformers pytorch bert text-classification en dataset:qqp license:apache-2.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #text-classification #en #dataset-qqp #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 

bert-base-uncased model fine-tuned on QQP
-----------------------------------------


This model was created using the nn\_pruning python library: the linear layers contains 36% of the original weights.


The model contains 50% of the original weights overall (the embeddings account for a significant part of the model, and they are not pruned by this method).



Fine-Pruning details
--------------------


This model was fine-tuned from the HuggingFace model checkpoint on task, and distilled from the model textattack/bert-base-uncased-QQP.
This model is case-insensitive: it does not make a difference between english and English.


A side-effect of block pruning is that some of the attention heads are completely removed: 54 heads were removed on a total of 144 (37.5%).



Details of the QQP dataset
--------------------------


Dataset: QQP, Split: train, # samples: 364K
Dataset: QQP, Split: eval, # samples: 40K


### Results


Pytorch model file size: '377MB' (original BERT: '420MB')




# samples: 364K
Dataset: QQP, Split: eval, # samples: 40K ### Results


Pytorch model file size: '377MB' (original BERT: '420MB')
TAGS
#transformers #pytorch #bert #text-classification #en #dataset-qqp #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
 # samples: 364K
Dataset: QQP, Split: eval, # samples: 40K ### Results


Pytorch model file size: '377MB' (original BERT: '420MB')
passage: TAGS
#transformers #pytorch #bert #text-classification #en #dataset-qqp #license-apache-2.0 #autotrain_compatible #endpoints_compatible #region-us 
# samples: 364K
Dataset: QQP, Split: eval, # samples: 40K### Results


Pytorch model file size: '377MB' (original BERT: '420MB')