
# SKEP-

## Introduction

SKEP (SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis) is proposed by Baidu in 2020,

SKEP propose Sentiment Knowledge Enhanced Pre-training for sentiment analysis. Sentiment masking and three sentiment pre-training objectives are designed to incorporate various types of knowledge for pre-training model.

More detail: https://aclanthology.org/2020.acl-main.374.pdf

## ⚠️ attention
Compared with the full version of the ernie_1.0_skep_large_ch, we lost the task_embeddings part in order to adapt to the Bert framework.



## Released Model Info

|Model Name|Language|Model Structure|
|:---:|:---:|:---:|
|skep-ernie1-bert-large| Chinese |Layer:24, Hidden:1024, Heads:24|

This released pytorch model is converted from the officially released PaddlePaddle SKEP model and 
a series of experiments have been conducted to check the accuracy of the conversion.

- Official PaddlePaddle SKEP repo: 
  1. https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/transformers/skep
  2. https://github.com/baidu/Senta
- Pytorch Conversion repo: Not released yet


## How to use
```Python
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("Yaxin/ernie_1.0_skep_large_ch")
model = AutoModel.from_pretrained("Yaxin/ernie_1.0_skep_large_ch")
```

## Citation

```bibtex
@article{tian2020skep,
  title={SKEP: Sentiment knowledge enhanced pre-training for sentiment analysis},
  author={Tian, Hao and Gao, Can and Xiao, Xinyan and Liu, Hao and He, Bolei and Wu, Hua and Wang, Haifeng and Wu, Feng},
  journal={arXiv preprint arXiv:2005.05635},
  year={2020}
}

```

```
reference:
https://github.com/nghuyong/ERNIE-Pytorch

```
{"language": "zh"}
transformers pytorch bert zh endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #zh #endpoints_compatible #region-us 

SKEP-
=====


Introduction
------------


SKEP (SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis) is proposed by Baidu in 2020,


SKEP propose Sentiment Knowledge Enhanced Pre-training for sentiment analysis. Sentiment masking and three sentiment pre-training objectives are designed to incorporate various types of knowledge for pre-training model.


More detail: URL


️ attention
-----------


Compared with the full version of the ernie\_1.0\_skep\_large\_ch, we lost the task\_embeddings part in order to adapt to the Bert framework.


Released Model Info
-------------------



This released pytorch model is converted from the officially released PaddlePaddle SKEP model and
a series of experiments have been conducted to check the accuracy of the conversion.


* Official PaddlePaddle SKEP repo:
	1. URL
	2. URL
* Pytorch Conversion repo: Not released yet


How to use
----------



TAGS
#transformers #pytorch #bert #zh #endpoints_compatible #region-us 

passage: TAGS
#transformers #pytorch #bert #zh #endpoints_compatible #region-us 
