# LIMIT-BERT

Code and model for the *EMNLP 2020 Findings* paper: 

[LIMIT-BERT: Linguistic Informed Multi-task BERT](https://arxiv.org/abs/1910.14296)) 

## Contents

1. [Requirements](#Requirements)
2. [Training](#Training)

## Requirements

* Python 3.6 or higher.
* Cython 0.25.2 or any compatible version.
* [PyTorch](http://pytorch.org/) 1.0.0+. 
* [EVALB](http://nlp.cs.nyu.edu/evalb/). Before starting, run `make` inside the `EVALB/` directory to compile an `evalb` executable. This will be called from Python for evaluation. 
* [pytorch-transformers](https://github.com/huggingface/pytorch-transformers) PyTorch 1.0.0+ or any compatible version.

#### Pre-trained Models (PyTorch)
The following pre-trained models are available for download from Google Drive:
* [`LIMIT-BERT`](https://drive.google.com/open?id=1fm0cK2A91iLG3lCpwowCCQSALnWS2X4i): 
  PyTorch version, same setting with BERT-Large-WWM，loading model with [pytorch-transformers](https://github.com/huggingface/pytorch-transformers).

## How to use

```
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("cooelf/limitbert")
model = AutoModel.from_pretrained("cooelf/limitbert")
```

Please see our original repo for the training scripts.

https://github.com/cooelf/LIMIT-BERT

## Training

To train LIMIT-BERT, simply run:
```
sh run_limitbert.sh
```
### Evaluation Instructions

To test after setting model path:
```
sh test_bert.sh
```

## Citation

```
@article{zhou2019limit,
  title={{LIMIT-BERT}: Linguistic informed multi-task {BERT}},
  author={Zhou, Junru and Zhang, Zhuosheng and Zhao, Hai},
  journal={arXiv preprint arXiv:1910.14296},
  year={2019}
}
```
{}
transformers pytorch arxiv:1910.14296 endpoints_compatible region:us
TAGS
#transformers #pytorch #arxiv-1910.14296 #endpoints_compatible #region-us 

# LIMIT-BERT

Code and model for the *EMNLP 2020 Findings* paper: 

LIMIT-BERT: Linguistic Informed Multi-task BERT) 

## Contents

1. Requirements
2. Training

## Requirements

* Python 3.6 or higher.
* Cython 0.25.2 or any compatible version.
* PyTorch 1.0.0+. 
* EVALB. Before starting, run 'make' inside the 'EVALB/' directory to compile an 'evalb' executable. This will be called from Python for evaluation. 
* pytorch-transformers PyTorch 1.0.0+ or any compatible version.

#### Pre-trained Models (PyTorch)
The following pre-trained models are available for download from Google Drive:
* 'LIMIT-BERT': 
  PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers.

## How to use



Please see our original repo for the training scripts.

URL

## Training

To train LIMIT-BERT, simply run:

### Evaluation Instructions

To test after setting model path:



# LIMIT-BERT

Code and model for the *EMNLP 2020 Findings* paper: 

LIMIT-BERT: Linguistic Informed Multi-task BERT) ## Contents

1. Requirements
2. Training ## Requirements

* Python 3.6 or higher.
* Cython 0.25.2 or any compatible version.
* PyTorch 1.0.0+. 
* EVALB. Before starting, run 'make' inside the 'EVALB/' directory to compile an 'evalb' executable. This will be called from Python for evaluation. 
* pytorch-transformers PyTorch 1.0.0+ or any compatible version. #### Pre-trained Models (PyTorch)
The following pre-trained models are available for download from Google Drive:
* 'LIMIT-BERT': 
  PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers. ## How to use



Please see our original repo for the training scripts.

URL ## Training

To train LIMIT-BERT, simply run: ### Evaluation Instructions

To test after setting model path:
TAGS
#transformers #pytorch #arxiv-1910.14296 #endpoints_compatible #region-us 
 # LIMIT-BERT

Code and model for the *EMNLP 2020 Findings* paper: 

LIMIT-BERT: Linguistic Informed Multi-task BERT) ## Contents

1. Requirements
2. Training ## Requirements

* Python 3.6 or higher.
* Cython 0.25.2 or any compatible version.
* PyTorch 1.0.0+. 
* EVALB. Before starting, run 'make' inside the 'EVALB/' directory to compile an 'evalb' executable. This will be called from Python for evaluation. 
* pytorch-transformers PyTorch 1.0.0+ or any compatible version. #### Pre-trained Models (PyTorch)
The following pre-trained models are available for download from Google Drive:
* 'LIMIT-BERT': 
  PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers. ## How to use



Please see our original repo for the training scripts.

URL ## Training

To train LIMIT-BERT, simply run: ### Evaluation Instructions

To test after setting model path:
passage: TAGS
#transformers #pytorch #arxiv-1910.14296 #endpoints_compatible #region-us 
# LIMIT-BERT

Code and model for the *EMNLP 2020 Findings* paper: 

LIMIT-BERT: Linguistic Informed Multi-task BERT)## Contents

1. Requirements
2. Training## Requirements

* Python 3.6 or higher.
* Cython 0.25.2 or any compatible version.
* PyTorch 1.0.0+. 
* EVALB. Before starting, run 'make' inside the 'EVALB/' directory to compile an 'evalb' executable. This will be called from Python for evaluation. 
* pytorch-transformers PyTorch 1.0.0+ or any compatible version.#### Pre-trained Models (PyTorch)
The following pre-trained models are available for download from Google Drive:
* 'LIMIT-BERT': 
  PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers.## How to use



Please see our original repo for the training scripts.

URL## Training

To train LIMIT-BERT, simply run:### Evaluation Instructions

To test after setting model path: