
## (BERT base) Language modeling in the legal domain in Portuguese (LeNER-Br)

**bert-base-cased-pt-lenerbr** is a Language Model in the legal domain in Portuguese that was finetuned on 20/12/2021 in Google Colab from the model [BERTimbau base](https://huggingface.co/neuralmind/bert-base-portuguese-cased) on the dataset [LeNER-Br language modeling](https://huggingface.co/datasets/pierreguillou/lener_br_finetuning_language_model) by using a MASK objective.

You can check as well the [version large of this model](https://huggingface.co/pierreguillou/bert-large-cased-pt-lenerbr).
  
## Blog post

This language model is used to get a NER model on the Portuguese judicial domain. You can check the fine-tuned NER model at [pierreguillou/ner-bert-base-cased-pt-lenerbr](https://huggingface.co/pierreguillou/ner-bert-base-cased-pt-lenerbr).

All informations and links are in this blog post: [NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro](https://medium.com/@pierre_guillou/nlp-modelos-e-web-app-para-reconhecimento-de-entidade-nomeada-ner-no-dom%C3%ADnio-jur%C3%ADdico-b658db55edfb) (29/12/2021)

## Widget & APP

You can test this model into the widget of this page.

## Using the model for inference in production
````
# install pytorch: check https://pytorch.org/
# !pip install transformers 
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("pierreguillou/bert-base-cased-pt-lenerbr")
model = AutoModelForMaskedLM.from_pretrained("pierreguillou/bert-base-cased-pt-lenerbr")
````

## Training procedure

## Notebook

The notebook of finetuning ([Finetuning_language_model_BERtimbau_LeNER_Br.ipynb](https://github.com/piegu/language-models/blob/master/Finetuning_language_model_BERtimbau_LeNER_Br.ipynb)) is in github.

### Training results

````
Num examples = 3227
Num Epochs = 5
Instantaneous batch size per device = 8
Total train batch size (w. parallel, distributed & accumulation) = 8
Gradient Accumulation steps = 1
Total optimization steps = 2020

Step	Training Loss	Validation Loss
100	 1.988700	     1.616412
200	 1.724900	     1.561100
300	 1.713400	     1.499991
400	 1.687400	     1.451414
500	 1.579700	     1.433665
600	 1.556900	     1.407338
700	 1.591400	     1.421942
800	 1.546000	     1.406395
900	 1.510100	     1.352389
1000	1.507100     	1.394799
1100	1.462200     	1.36809373471
````
{"language": ["pt"], "tags": ["generated_from_trainer"], "datasets": ["pierreguillou/lener_br_finetuning_language_model"], "widget": [{"text": "Com efeito, se tal fosse poss\u00edvel, o Poder [MASK] \u2013 que n\u00e3o disp\u00f5e de fun\u00e7\u00e3o legislativa \u2013 passaria a desempenhar atribui\u00e7\u00e3o que lhe \u00e9 institucionalmente estranha (a de legislador positivo), usurpando, desse modo, no contexto de um sistema de poderes essencialmente limitados, compet\u00eancia que n\u00e3o lhe pertence, com evidente transgress\u00e3o ao princ\u00edpio constitucional da separa\u00e7\u00e3o de poderes."}], "model-index": [{"name": "checkpoints", "results": [{"task": {"type": "fill-mask", "name": "Fill Mask"}, "dataset": {"name": "pierreguillou/lener_br_finetuning_language_model", "type": "pierreguillou/lener_br_finetuning_language_model"}, "metrics": [{"type": "loss", "value": 1.352389, "name": "Loss"}]}]}]}
transformers pytorch bert fill-mask generated_from_trainer pt dataset:pierreguillou/lener_br_finetuning_language_model model-index autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #generated_from_trainer #pt #dataset-pierreguillou/lener_br_finetuning_language_model #model-index #autotrain_compatible #endpoints_compatible #region-us 


## (BERT base) Language modeling in the legal domain in Portuguese (LeNER-Br)

bert-base-cased-pt-lenerbr is a Language Model in the legal domain in Portuguese that was finetuned on 20/12/2021 in Google Colab from the model BERTimbau base on the dataset LeNER-Br language modeling by using a MASK objective.

You can check as well the version large of this model.
  
## Blog post

This language model is used to get a NER model on the Portuguese judicial domain. You can check the fine-tuned NER model at pierreguillou/ner-bert-base-cased-pt-lenerbr.

All informations and links are in this blog post: NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro (29/12/2021)

## Widget & APP

You can test this model into the widget of this page.

## Using the model for inference in production
'

## Training procedure

## Notebook

The notebook of finetuning (Finetuning_language_model_BERtimbau_LeNER_Br.ipynb) is in github.

### Training results

'
## (BERT base) Language modeling in the legal domain in Portuguese (LeNER-Br)

bert-base-cased-pt-lenerbr is a Language Model in the legal domain in Portuguese that was finetuned on 20/12/2021 in Google Colab from the model BERTimbau base on the dataset LeNER-Br language modeling by using a MASK objective.

You can check as well the version large of this model. ## Blog post

This language model is used to get a NER model on the Portuguese judicial domain. You can check the fine-tuned NER model at pierreguillou/ner-bert-base-cased-pt-lenerbr.

All informations and links are in this blog post: NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro (29/12/2021) ## Widget & APP

You can test this model into the widget of this page. ## Using the model for inference in production
' ## Training procedure ## Notebook

The notebook of finetuning (Finetuning_language_model_BERtimbau_LeNER_Br.ipynb) is in github. ### Training results

'
TAGS
#transformers #pytorch #bert #fill-mask #generated_from_trainer #pt #dataset-pierreguillou/lener_br_finetuning_language_model #model-index #autotrain_compatible #endpoints_compatible #region-us 
 ## (BERT base) Language modeling in the legal domain in Portuguese (LeNER-Br)

bert-base-cased-pt-lenerbr is a Language Model in the legal domain in Portuguese that was finetuned on 20/12/2021 in Google Colab from the model BERTimbau base on the dataset LeNER-Br language modeling by using a MASK objective.

You can check as well the version large of this model. ## Blog post

This language model is used to get a NER model on the Portuguese judicial domain. You can check the fine-tuned NER model at pierreguillou/ner-bert-base-cased-pt-lenerbr.

All informations and links are in this blog post: NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro (29/12/2021) ## Widget & APP

You can test this model into the widget of this page. ## Using the model for inference in production
' ## Training procedure ## Notebook

The notebook of finetuning (Finetuning_language_model_BERtimbau_LeNER_Br.ipynb) is in github. ### Training results

'
passage: TAGS
#transformers #pytorch #bert #fill-mask #generated_from_trainer #pt #dataset-pierreguillou/lener_br_finetuning_language_model #model-index #autotrain_compatible #endpoints_compatible #region-us 
## (BERT base) Language modeling in the legal domain in Portuguese (LeNER-Br)

bert-base-cased-pt-lenerbr is a Language Model in the legal domain in Portuguese that was finetuned on 20/12/2021 in Google Colab from the model BERTimbau base on the dataset LeNER-Br language modeling by using a MASK objective.

You can check as well the version large of this model.## Blog post

This language model is used to get a NER model on the Portuguese judicial domain. You can check the fine-tuned NER model at pierreguillou/ner-bert-base-cased-pt-lenerbr.

All informations and links are in this blog post: NLP | Modelos e Web App para Reconhecimento de Entidade Nomeada (NER) no domínio jurídico brasileiro (29/12/2021)## Widget & APP

You can test this model into the widget of this page.## Using the model for inference in production
'## Training procedure## Notebook

The notebook of finetuning (Finetuning_language_model_BERtimbau_LeNER_Br.ipynb) is in github.### Training results

'