
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# ukrainian-qa

This model is a fine-tuned version of [ukr-models/xlm-roberta-base-uk](https://huggingface.co/ukr-models/xlm-roberta-base-uk) on the [UA-SQuAD](https://github.com/fido-ai/ua-datasets/tree/main/ua_datasets/src/question_answering) dataset.

Link to training scripts - [https://github.com/robinhad/ukrainian-qa](https://github.com/robinhad/ukrainian-qa)  
It achieves the following results on the evaluation set:
- Loss: 1.4778

## Model description

More information needed

## How to use

```python
from transformers import pipeline, AutoTokenizer, AutoModelForQuestionAnswering
model_name = "robinhad/ukrainian-qa"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForQuestionAnswering.from_pretrained(model_name)

qa_model = pipeline("question-answering", model=model.to("cpu"), tokenizer=tokenizer)
question = "Де ти живеш?"
context = "Мене звати Сара і я живу у Лондоні"
qa_model(question = question, context = context)
```

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.4526        | 1.0   | 650  | 1.3631          |
| 1.3317        | 2.0   | 1300 | 1.2229          |
| 1.0693        | 3.0   | 1950 | 1.2184          |
| 0.6851        | 4.0   | 2600 | 1.3171          |
| 0.5594        | 5.0   | 3250 | 1.3893          |
| 0.4954        | 6.0   | 3900 | 1.4778          |


### Framework versions

- Transformers 4.19.2
- Pytorch 1.11.0
- Datasets 2.2.2
- Tokenizers 0.12.1

{"language": "uk", "license": "mit", "tags": ["generated_from_trainer"], "widget": [{"text": "\u0429\u043e \u0432\u0456\u0434\u043f\u0440\u0430\u0432\u043b\u044f\u0442\u044c \u0434\u043b\u044f \u0417\u0421\u0423?", "context": "\u041f\u0440\u043e \u0446\u0435 \u043f\u043e\u0432\u0456\u0434\u043e\u043c\u0438\u0432 \u043c\u0456\u043d\u0456\u0441\u0442\u0440 \u043e\u0431\u043e\u0440\u043e\u043d\u0438 \u0410\u0440\u0432\u0456\u0434\u0430\u0441 \u0410\u043d\u0443\u0448\u0430\u0443\u0441\u043a\u0430\u0441. \u0423\u0440\u044f\u0434 \u041b\u0438\u0442\u0432\u0438 \u043d\u0435 \u043c\u0430\u0454 \u043d\u0430\u043c\u0456\u0440\u0443 \u0437\u0443\u043f\u0438\u043d\u044f\u0442\u0438\u0441\u044f \u0443 \u0432\u0456\u0439\u0441\u044c\u043a\u043e\u0432\u043e-\u0442\u0435\u0445\u043d\u0456\u0447\u043d\u0456\u0439 \u0434\u043e\u043f\u043e\u043c\u043e\u0437\u0456 \u0423\u043a\u0440\u0430\u0457\u043d\u0456. \u0417\u0431\u0440\u043e\u0439\u043d\u0456 \u0441\u0438\u043b\u0438 \u043e\u0442\u0440\u0438\u043c\u0430\u044e\u0442\u044c \u0430\u043d\u0442\u0438\u0434\u0440\u043e\u043d\u0438, \u0442\u0435\u043f\u043b\u043e\u0432\u0456\u0437\u043e\u0440\u0438 \u0442\u0430 \u0443\u0434\u0430\u0440\u043d\u0438\u0439 \u0431\u0435\u0437\u043f\u0456\u043b\u043e\u0442\u043d\u0438\u043a. \u00ab\u041d\u0435\u0437\u0430\u0431\u0430\u0440\u043e\u043c \u041b\u0438\u0442\u0432\u0430 \u043f\u0435\u0440\u0435\u0434\u0430\u0441\u0442\u044c \u0423\u043a\u0440\u0430\u0457\u043d\u0456 \u043d\u0435 \u043b\u0438\u0448\u0435 \u043e\u0431\u0456\u0446\u044f\u043d\u0456 \u0431\u0440\u043e\u043d\u0435\u0442\u0435\u0445\u043d\u0456\u043a\u0443, \u0432\u0430\u043d\u0442\u0430\u0436\u0456\u0432\u043a\u0438 \u0442\u0430 \u043f\u043e\u0437\u0430\u0448\u043b\u044f\u0445\u043e\u0432\u0438\u043a\u0438, \u0430\u043b\u0435 \u0442\u0430\u043a\u043e\u0436 \u043d\u043e\u0432\u0443 \u043f\u0430\u0440\u0442\u0456\u044e \u0430\u043d\u0442\u0438\u0434\u0440\u043e\u043d\u0456\u0432 \u0442\u0430 \u0442\u0435\u043f\u043b\u043e\u0432\u0456\u0437\u043e\u0440\u0456\u0432. \u0406, \u0437\u0432\u0438\u0447\u0430\u0439\u043d\u043e, \u0411\u0430\u0439\u0440\u0430\u043a\u0442\u0430\u0440, \u044f\u043a\u0438\u0439 \u043f\u0440\u0438\u0434\u0431\u0430\u044e\u0442\u044c \u043d\u0430 \u0437\u0456\u0431\u0440\u0430\u043d\u0456 \u043b\u0438\u0442\u043e\u0432\u0446\u044f\u043c\u0438 \u0433\u0440\u043e\u0448\u0456\u00bb, - \u043d\u0430\u043f\u0438\u0441\u0430\u0432 \u0433\u043b\u0430\u0432\u0430 \u041c\u0456\u043d\u043e\u0431\u043e\u0440\u043e\u043d\u0438."}], "model-index": [{"name": "ukrainian-qa", "results": []}]}
transformers pytorch tensorboard xlm-roberta question-answering generated_from_trainer uk license:mit endpoints_compatible region:us
TAGS
#transformers #pytorch #tensorboard #xlm-roberta #question-answering #generated_from_trainer #uk #license-mit #endpoints_compatible #region-us 

ukrainian-qa
============


This model is a fine-tuned version of ukr-models/xlm-roberta-base-uk on the UA-SQuAD dataset.


Link to training scripts - URL  

It achieves the following results on the evaluation set:


* Loss: 1.4778


Model description
-----------------


More information needed


How to use
----------


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 6


### Training results



### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0
* Datasets 2.2.2
* Tokenizers 0.12.1



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 6 ### Training results ### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0
* Datasets 2.2.2
* Tokenizers 0.12.1
TAGS
#transformers #pytorch #tensorboard #xlm-roberta #question-answering #generated_from_trainer #uk #license-mit #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 6 ### Training results ### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0
* Datasets 2.2.2
* Tokenizers 0.12.1
passage: TAGS
#transformers #pytorch #tensorboard #xlm-roberta #question-answering #generated_from_trainer #uk #license-mit #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 2e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr\_scheduler\_type: linear
* num\_epochs: 6### Training results### Framework versions


* Transformers 4.19.2
* Pytorch 1.11.0
* Datasets 2.2.2
* Tokenizers 0.12.1