# ERNIE-2.0-large

## Introduction

ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
which builds and learns incrementally pre-training tasks through constant multi-task learning. 
Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 

More detail: https://arxiv.org/abs/1907.12412

## Released Model Info

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE
- Pytorch Conversion repo:  https://github.com/nghuyong/ERNIE-Pytorch

## How to use
```Python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-2.0-large-en")
model = AutoModel.from_pretrained("nghuyong/ernie-2.0-large-en")
```

## Citation

```bibtex
@article{sun2019ernie20,
  title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding},
  author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng},
  journal={arXiv preprint arXiv:1907.12412},
  year={2019} 
}
```

{}
transformers pytorch ernie feature-extraction arxiv:1907.12412 endpoints_compatible has_space region:us
TAGS
#transformers #pytorch #ernie #feature-extraction #arxiv-1907.12412 #endpoints_compatible #has_space #region-us 

# ERNIE-2.0-large

## Introduction

ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
which builds and learns incrementally pre-training tasks through constant multi-task learning. 
Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 

More detail: URL

## Released Model Info

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle ERNIE repo: URL
- Pytorch Conversion repo:  URL

## How to use



# ERNIE-2.0-large ## Introduction

ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
which builds and learns incrementally pre-training tasks through constant multi-task learning. 
Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 

More detail: URL ## Released Model Info

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle ERNIE repo: URL
- Pytorch Conversion repo:  URL ## How to use
TAGS
#transformers #pytorch #ernie #feature-extraction #arxiv-1907.12412 #endpoints_compatible #has_space #region-us 
 # ERNIE-2.0-large ## Introduction

ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
which builds and learns incrementally pre-training tasks through constant multi-task learning. 
Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 

More detail: URL ## Released Model Info

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle ERNIE repo: URL
- Pytorch Conversion repo:  URL ## How to use
passage: TAGS
#transformers #pytorch #ernie #feature-extraction #arxiv-1907.12412 #endpoints_compatible #has_space #region-us 
# ERNIE-2.0-large## Introduction

ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, 
which builds and learns incrementally pre-training tasks through constant multi-task learning. 
Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. 

More detail: URL## Released Model Info

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle ERNIE repo: URL
- Pytorch Conversion repo:  URL## How to use