
# XLM-RoBERTa Large trained on Dravidian Language QA

## Overview
**Language model:** XLM-RoBERTa-lg
**Language:** Multilingual, focussed on Tamil & Hindi 
**Downstream-task:** Extractive QA
**Eval data:** K-Fold on Training Data

## Hyperparameters
```
batch_size = 4
base_LM_model = "xlm-roberta-large"
learning_rate = 1e-5

optimizer = AdamW
weight_decay = 1e-2
epsilon = 1e-8
max_grad_norm = 1.0

lr_schedule = LinearWarmup
warmup_proportion = 0.2

max_seq_len = 256
doc_stride=128
max_query_length=64
``` 

## Performance
Evaluated on our human annotated dataset with 1000 tamil question-context pairs [link]
```
  "em": 77.536,
  "f1": 85.665
```

## Usage
### In Transformers
```python
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = "Srini99/FYP_TamilQA"

model = AutoModelForQuestionAnswering.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
    'question': 'யாரால் பொங்கல் சிறப்பாகக் கொண்டாடப்படுகிறது?',
    'context': 'பொங்கல் என்பது தமிழர்களால் சிறப்பாகக் கொண்டாடப்படும் ஓர் அறுவடைப் பண்டிகை ஆகும்.'
}
res = nlp(QA_input)
```
{"language": ["multilingual", "ta"], "tags": ["question-answering"], "datasets": ["squad_v2", "chaii", "mlqa", "xquad"], "metrics": ["Exact Match", "F1"], "widget": [{"text": "\u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8\u0baf\u0bbf\u0bb2\u0bcd \u0b8e\u0ba4\u0bcd\u0ba4\u0ba9\u0bc8 \u0bae\u0b95\u0bcd\u0b95\u0bb3\u0bcd \u0bb5\u0bbe\u0bb4\u0bcd\u0b95\u0bbf\u0ba9\u0bcd\u0bb1\u0ba9\u0bb0\u0bcd?", "context": "\u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8 (Chennai) \u0ba4\u0bae\u0bbf\u0bb4\u0bcd\u0ba8\u0bbe\u0b9f\u0bcd\u0b9f\u0bbf\u0ba9\u0bcd \u0ba4\u0bb2\u0bc8\u0ba8\u0b95\u0bb0\u0bae\u0bc1\u0bae\u0bcd \u0b87\u0ba8\u0bcd\u0ba4\u0bbf\u0baf\u0bbe\u0bb5\u0bbf\u0ba9\u0bcd \u0ba8\u0bbe\u0ba9\u0bcd\u0b95\u0bbe\u0bb5\u0ba4\u0bc1 \u0baa\u0bc6\u0bb0\u0bbf\u0baf \u0ba8\u0b95\u0bb0\u0bae\u0bc1\u0bae\u0bcd \u0b86\u0b95\u0bc1\u0bae\u0bcd. 1996 \u0b86\u0bae\u0bcd \u0b86\u0ba3\u0bcd\u0b9f\u0bc1\u0b95\u0bcd\u0b95\u0bc1 \u0bae\u0bc1\u0ba9\u0bcd\u0ba9\u0bb0\u0bcd \u0b87\u0ba8\u0bcd\u0ba8\u0b95\u0bb0\u0bae\u0bcd \u0bae\u0bc6\u0b9f\u0bcd\u0bb0\u0bbe\u0bb8\u0bcd (Madras) \u0b8e\u0ba9\u0bcd\u0bb1\u0bc1 \u0b85\u0bb4\u0bc8\u0b95\u0bcd\u0b95\u0baa\u0bcd\u0baa\u0b9f\u0bcd\u0b9f\u0bc1 \u0bb5\u0ba8\u0bcd\u0ba4\u0ba4\u0bc1. \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8, \u0bb5\u0b99\u0bcd\u0b95\u0bbe\u0bb3 \u0bb5\u0bbf\u0bb0\u0bbf\u0b95\u0bc1\u0b9f\u0bbe\u0bb5\u0bbf\u0ba9\u0bcd \u0b95\u0bb0\u0bc8\u0baf\u0bbf\u0bb2\u0bcd \u0b85\u0bae\u0bc8\u0ba8\u0bcd\u0ba4 \u0ba4\u0bc1\u0bb1\u0bc8\u0bae\u0bc1\u0b95 \u0ba8\u0b95\u0bb0\u0b99\u0bcd\u0b95\u0bb3\u0bc1\u0bb3\u0bcd \u0b92\u0ba9\u0bcd\u0bb1\u0bc1. \u0b9a\u0bc1\u0bae\u0bbe\u0bb0\u0bcd 10 \u0bae\u0bbf\u0bb2\u0bcd\u0bb2\u0bbf\u0baf\u0ba9\u0bcd (\u0b92\u0bb0\u0bc1 \u0b95\u0bcb\u0b9f\u0bbf) \u0bae\u0b95\u0bcd\u0b95\u0bb3\u0bcd \u0bb5\u0bbe\u0bb4\u0bc1\u0bae\u0bcd \u0b87\u0ba8\u0bcd\u0ba8\u0b95\u0bb0\u0bae\u0bcd, \u0b89\u0bb2\u0b95\u0bbf\u0ba9\u0bcd 35 \u0baa\u0bc6\u0bb0\u0bbf\u0baf \u0bae\u0bbe\u0ba8\u0b95\u0bb0\u0b99\u0bcd\u0b95\u0bb3\u0bc1\u0bb3\u0bcd \u0b92\u0ba9\u0bcd\u0bb1\u0bc1. 17\u0b86\u0bae\u0bcd \u0ba8\u0bc2\u0bb1\u0bcd\u0bb1\u0bbe\u0ba3\u0bcd\u0b9f\u0bbf\u0bb2\u0bcd \u0b86\u0b99\u0bcd\u0b95\u0bbf\u0bb2\u0bc7\u0baf\u0bb0\u0bcd \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8\u0baf\u0bbf\u0bb2\u0bcd \u0b95\u0bbe\u0bb2\u0bcd \u0baa\u0ba4\u0bbf\u0ba4\u0bcd\u0ba4\u0ba4\u0bc1 \u0bae\u0bc1\u0ba4\u0bb2\u0bcd, \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8 \u0ba8\u0b95\u0bb0\u0bae\u0bcd \u0b92\u0bb0\u0bc1 \u0bae\u0bc1\u0b95\u0bcd\u0b95\u0bbf\u0baf \u0ba8\u0b95\u0bb0\u0bae\u0bbe\u0b95 \u0bb5\u0bb3\u0bb0\u0bcd\u0ba8\u0bcd\u0ba4\u0bc1 \u0bb5\u0ba8\u0bcd\u0ba4\u0bbf\u0bb0\u0bc1\u0b95\u0bcd\u0b95\u0bbf\u0bb1\u0ba4\u0bc1. \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8 \u0ba4\u0bc6\u0ba9\u0bcd\u0ba9\u0bbf\u0ba8\u0bcd\u0ba4\u0bbf\u0baf\u0bbe\u0bb5\u0bbf\u0ba9\u0bcd \u0bb5\u0bbe\u0b9a\u0bb2\u0bbe\u0b95\u0b95\u0bcd \u0b95\u0bb0\u0bc1\u0ba4\u0baa\u0bcd\u0baa\u0b9f\u0bc1\u0b95\u0bbf\u0bb1\u0ba4\u0bc1. \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8 \u0ba8\u0b95\u0bb0\u0bbf\u0bb2\u0bcd \u0b89\u0bb3\u0bcd\u0bb3 \u0bae\u0bc6\u0bb0\u0bbf\u0ba9\u0bbe \u0b95\u0b9f\u0bb1\u0bcd\u0b95\u0bb0\u0bc8 \u0b89\u0bb2\u0b95\u0bbf\u0ba9\u0bcd \u0ba8\u0bc0\u0bb3\u0bae\u0bbe\u0ba9 \u0b95\u0b9f\u0bb1\u0bcd\u0b95\u0bb0\u0bc8\u0b95\u0bb3\u0bc1\u0bb3\u0bcd \u0b92\u0ba9\u0bcd\u0bb1\u0bc1. \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8 \u0b95\u0bcb\u0bb2\u0bbf\u0bb5\u0bc1\u0b9f\u0bcd (Kollywood) \u0b8e\u0ba9 \u0b85\u0bb1\u0bbf\u0baf\u0baa\u0bcd\u0baa\u0b9f\u0bc1\u0bae\u0bcd \u0ba4\u0bae\u0bbf\u0bb4\u0bcd\u0ba4\u0bcd \u0ba4\u0bbf\u0bb0\u0bc8\u0baa\u0bcd\u0baa\u0b9f\u0ba4\u0bcd \u0ba4\u0bc1\u0bb1\u0bc8\u0baf\u0bbf\u0ba9\u0bcd \u0ba4\u0bbe\u0baf\u0b95\u0bae\u0bcd. \u0baa\u0bb2 \u0bb5\u0bbf\u0bb3\u0bc8\u0baf\u0bbe\u0b9f\u0bcd\u0b9f\u0bc1 \u0b85\u0bb0\u0b99\u0bcd\u0b95\u0b99\u0bcd\u0b95\u0bb3\u0bcd \u0b89\u0bb3\u0bcd\u0bb3 \u0b9a\u0bc6\u0ba9\u0bcd\u0ba9\u0bc8\u0baf\u0bbf\u0bb2\u0bcd \u0baa\u0bb2 \u0bb5\u0bbf\u0bb3\u0bc8\u0baf\u0bbe\u0b9f\u0bcd\u0b9f\u0bc1\u0baa\u0bcd \u0baa\u0bcb\u0b9f\u0bcd\u0b9f\u0bbf\u0b95\u0bb3\u0bc1\u0bae\u0bcd \u0ba8\u0b9f\u0bc8\u0baa\u0bc6\u0bb1\u0bc1\u0b95\u0bbf\u0ba9\u0bcd\u0bb1\u0ba9.", "example_title": "Question Answering"}]}
transformers pytorch xlm-roberta question-answering multilingual ta dataset:squad_v2 dataset:chaii dataset:mlqa dataset:xquad endpoints_compatible region:us
TAGS
#transformers #pytorch #xlm-roberta #question-answering #multilingual #ta #dataset-squad_v2 #dataset-chaii #dataset-mlqa #dataset-xquad #endpoints_compatible #region-us 


# XLM-RoBERTa Large trained on Dravidian Language QA

## Overview
Language model: XLM-RoBERTa-lg
Language: Multilingual, focussed on Tamil & Hindi 
Downstream-task: Extractive QA
Eval data: K-Fold on Training Data

## Hyperparameters
 

## Performance
Evaluated on our human annotated dataset with 1000 tamil question-context pairs [link]


## Usage
### In Transformers

# XLM-RoBERTa Large trained on Dravidian Language QA ## Overview
Language model: XLM-RoBERTa-lg
Language: Multilingual, focussed on Tamil & Hindi 
Downstream-task: Extractive QA
Eval data: K-Fold on Training Data ## Hyperparameters ## Performance
Evaluated on our human annotated dataset with 1000 tamil question-context pairs [link] ## Usage ### In Transformers
TAGS
#transformers #pytorch #xlm-roberta #question-answering #multilingual #ta #dataset-squad_v2 #dataset-chaii #dataset-mlqa #dataset-xquad #endpoints_compatible #region-us 
 # XLM-RoBERTa Large trained on Dravidian Language QA ## Overview
Language model: XLM-RoBERTa-lg
Language: Multilingual, focussed on Tamil & Hindi 
Downstream-task: Extractive QA
Eval data: K-Fold on Training Data ## Hyperparameters ## Performance
Evaluated on our human annotated dataset with 1000 tamil question-context pairs [link] ## Usage ### In Transformers
passage: TAGS
#transformers #pytorch #xlm-roberta #question-answering #multilingual #ta #dataset-squad_v2 #dataset-chaii #dataset-mlqa #dataset-xquad #endpoints_compatible #region-us 
# XLM-RoBERTa Large trained on Dravidian Language QA## Overview
Language model: XLM-RoBERTa-lg
Language: Multilingual, focussed on Tamil & Hindi 
Downstream-task: Extractive QA
Eval data: K-Fold on Training Data## Hyperparameters## Performance
Evaluated on our human annotated dataset with 1000 tamil question-context pairs [link]## Usage### In Transformers