# mobilebert-uncased-finetuned-squadv1

This model is a finetuned version of the [mobilebert-uncased](https://huggingface.co/google/mobilebert-uncased/tree/main) model on the SQuADv1 task.
To make this TPU-trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

It is produced as part of the work on the paper [The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models](https://arxiv.org/abs/2203.07259).

SQuADv1 dev-set:
```
EM = 83.96
F1 = 90.90
```

Code: [https://github.com/neuralmagic/sparseml/tree/main/research/optimal_BERT_surgeon_oBERT](https://github.com/neuralmagic/sparseml/tree/main/research/optimal_BERT_surgeon_oBERT)

If you find the model useful, please consider citing our work.

## Citation info
```bibtex
@article{kurtic2022optimal,
  title={The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models},
  author={Kurtic, Eldar and Campos, Daniel and Nguyen, Tuan and Frantar, Elias and Kurtz, Mark and Fineran, Benjamin and Goin, Michael and Alistarh, Dan},
  journal={arXiv preprint arXiv:2203.07259},
  year={2022}
}
```


{"language": "en", "tags": ["bert", "mobilebert", "oBERT"], "datasets": "squad"}
transformers pytorch mobilebert question-answering bert oBERT en dataset:squad arxiv:2203.07259 endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #mobilebert #question-answering #bert #oBERT #en #dataset-squad #arxiv-2203.07259 #endpoints_compatible #has_space #region-us 

# mobilebert-uncased-finetuned-squadv1

This model is a finetuned version of the mobilebert-uncased model on the SQuADv1 task.
To make this TPU-trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

It is produced as part of the work on the paper The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models.

SQuADv1 dev-set:


Code: URL

If you find the model useful, please consider citing our work.

info



# mobilebert-uncased-finetuned-squadv1

This model is a finetuned version of the mobilebert-uncased model on the SQuADv1 task.
To make this TPU-trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

It is produced as part of the work on the paper The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models.

SQuADv1 dev-set:


Code: URL

If you find the model useful, please consider citing our work.

info
TAGS
#transformers #pytorch #mobilebert #question-answering #bert #oBERT #en #dataset-squad #arxiv-2203.07259 #endpoints_compatible #has_space #region-us 
 # mobilebert-uncased-finetuned-squadv1

This model is a finetuned version of the mobilebert-uncased model on the SQuADv1 task.
To make this TPU-trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

It is produced as part of the work on the paper The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models.

SQuADv1 dev-set:


Code: URL

If you find the model useful, please consider citing our work.

info
passage: TAGS
#transformers #pytorch #mobilebert #question-answering #bert #oBERT #en #dataset-squad #arxiv-2203.07259 #endpoints_compatible #has_space #region-us 
# mobilebert-uncased-finetuned-squadv1

This model is a finetuned version of the mobilebert-uncased model on the SQuADv1 task.
To make this TPU-trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

It is produced as part of the work on the paper The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models.

SQuADv1 dev-set:


Code: URL

If you find the model useful, please consider citing our work.

info