# roberta-base for MLM 

Objective: To make a Roberta Base for the Movie Domain by using various Movie Datasets as simple text for Masked Language Modeling. 
  This is the Movie Roberta to be used in Movie Domain applications. 
```
model_name = "thatdramebaazguy/movie-roberta-base"
pipeline(model=model_name, tokenizer=model_name, revision="v1.0", task="Fill-Mask")
```
## Overview
**Language model:** roberta-base  
**Language:** English  
**Downstream-task:** Fill-Mask  
**Training data:** imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names  
**Eval data:** imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names    
**Infrastructure**: 4x Tesla v100   
**Code:**  See [example](https://github.com/adityaarunsinghal/Domain-Adaptation/blob/master/scripts/shell_scripts/train_movie_roberta.sh)    

## Hyperparameters
```
Num examples = 4767233
Num Epochs = 2
Instantaneous batch size per device = 20
Total train batch size (w. parallel, distributed & accumulation) = 80
Gradient Accumulation steps = 1
Total optimization steps = 119182
eval_loss  = 1.6153
eval_samples = 20573
perplexity = 5.0296
learning_rate=5e-05
n_gpu = 4

``` 
## Performance

perplexity = 5.0296

Some of my work: 
- [Domain-Adaptation Project](https://github.com/adityaarunsinghal/Domain-Adaptation/)

---

{"language": ["English"], "license": "cc-by-4.0", "tags": ["roberta", "roberta-base", "masked-language-modeling", "masked-lm"], "datasets": ["imdb", "cornell_movie_dialogue", "polarity_movie_data", "25mlens_movie_data"]}
transformers pytorch tf jax roberta fill-mask roberta-base masked-language-modeling masked-lm dataset:imdb dataset:cornell_movie_dialogue dataset:polarity_movie_data dataset:25mlens_movie_data license:cc-by-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #tf #jax #roberta #fill-mask #roberta-base #masked-language-modeling #masked-lm #dataset-imdb #dataset-cornell_movie_dialogue #dataset-polarity_movie_data #dataset-25mlens_movie_data #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 

# roberta-base for MLM 

Objective: To make a Roberta Base for the Movie Domain by using various Movie Datasets as simple text for Masked Language Modeling. 
  This is the Movie Roberta to be used in Movie Domain applications. 

## Overview
Language model: roberta-base  
Language: English  
Downstream-task: Fill-Mask  
Training data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names  
Eval data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names    
Infrastructure: 4x Tesla v100   
Code:  See example    

## Hyperparameters
 
## Performance

perplexity = 5.0296

Some of my work: 
- Domain-Adaptation Project

---

# roberta-base for MLM 

Objective: To make a Roberta Base for the Movie Domain by using various Movie Datasets as simple text for Masked Language Modeling. 
  This is the Movie Roberta to be used in Movie Domain applications. ## Overview
Language model: roberta-base  
Language: English  
Downstream-task: Fill-Mask  
Training data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names  
Eval data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names    
Infrastructure: 4x Tesla v100   
Code:  See example ## Hyperparameters ## Performance

perplexity = 5.0296

Some of my work: 
- Domain-Adaptation Project

---
TAGS
#transformers #pytorch #tf #jax #roberta #fill-mask #roberta-base #masked-language-modeling #masked-lm #dataset-imdb #dataset-cornell_movie_dialogue #dataset-polarity_movie_data #dataset-25mlens_movie_data #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 # roberta-base for MLM 

Objective: To make a Roberta Base for the Movie Domain by using various Movie Datasets as simple text for Masked Language Modeling. 
  This is the Movie Roberta to be used in Movie Domain applications. ## Overview
Language model: roberta-base  
Language: English  
Downstream-task: Fill-Mask  
Training data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names  
Eval data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names    
Infrastructure: 4x Tesla v100   
Code:  See example ## Hyperparameters ## Performance

perplexity = 5.0296

Some of my work: 
- Domain-Adaptation Project

---
passage: TAGS
#transformers #pytorch #tf #jax #roberta #fill-mask #roberta-base #masked-language-modeling #masked-lm #dataset-imdb #dataset-cornell_movie_dialogue #dataset-polarity_movie_data #dataset-25mlens_movie_data #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 
# roberta-base for MLM 

Objective: To make a Roberta Base for the Movie Domain by using various Movie Datasets as simple text for Masked Language Modeling. 
  This is the Movie Roberta to be used in Movie Domain applications.## Overview
Language model: roberta-base  
Language: English  
Downstream-task: Fill-Mask  
Training data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names  
Eval data: imdb, polarity movie data, cornell_movie_dialogue, 25mlens movie names    
Infrastructure: 4x Tesla v100   
Code:  See example## Hyperparameters## Performance

perplexity = 5.0296

Some of my work: 
- Domain-Adaptation Project

---