

# Estonian NER model based on EstBERT

This model is a fine-tuned version of [tartuNLP/EstBERT](https://huggingface.co/tartuNLP/EstBERT) on the Estonian NER dataset. The model was trained by tartuNLP, the NLP research group at the institute of Computer Science at the University of Tartu.

It achieves the following results on the test set:

- Loss: 0.3565  
- Precision: 0.7612  
- Recall: 0.7744  
- F1: 0.7678  
- Accuracy: 0.9672  

The entity-level results are as follows:


|         | Precision | Recall  | F1      | Number  |
|---------| --------- | ------- | ------- | ------- |
| DATE    | 0.7278    | 0.7258  | 0.7268  | 372     |
| EVENT   | 0.3721    | 0.5714  | 0.4507  | 28      |
| GPE     | 0.8679    | 0.8369  | 0.8521  | 840     |
| LOC     | 0.6545    | 0.4832  | 0.5560  | 149     |
| MONEY   | 0.6625    | 0.6023  | 0.6310  | 88      |
| ORG     | 0.6761    | 0.7267  | 0.7005  | 589     |
| PER     | 0.8255    | 0.9068  | 0.8642  | 751     |
| PERCENT | 1.0       | 0.9589  | 0.9790  | 73      |
| PROD    | 0.6030    | 0.5430  | 0.5714  | 221     |
| TIME    | 0.5682    | 0.5556  | 0.5618  | 45      |
| TITLE   | 0.7       | 0.8063  | 0.7494  | 191     |

## How to use 

You can use this model with Transformers pipeline for NER. Post-processing of results may be necessary as the model occasionally tags subword tokens as entities. 

```
from transformers import BertTokenizer, BertForTokenClassification
from transformers import pipeline

tokenizer = BertTokenizer.from_pretrained('tartuNLP/EstBERT_NER')
bertner = BertForTokenClassification.from_pretrained('tartuNLP/EstBERT_NER')

nlp = pipeline("ner", model=bertner, tokenizer=tokenizer)

text = "Kaia Kanepi (WTA 57.) langes USA-s Charlestonis toimuval WTA 500 kategooria tenniseturniiril konkurentsist kaheksandikfinaalis, kaotades poolatarile Magda Linette'ile (WTA 64.) 3 : 6, 6 : 4, 2 : 6."

ner_results = nlp(text)

tokens=tokenizer(text)
tokens=tokenizer.convert_ids_to_tokens(tokens['input_ids'])


print(f'tokens: {tokens}')
print(f'NER model:{ner_results}')

```
```
tokens: ['[CLS]', 'kai', '##a', 'kanepi', '(', 'w', '##ta', '57', '.', ')', 'langes', 'usa', '-', 's', 'cha', '##rl', '##est', '##onis', 'toimuval', 'w', '##ta', '500', 'kategooria', 'tennise', '##turniiril', 'konkurentsist', 'kaheksandik', '##finaalis', ',', 'kaotades', 'poola', '##tari', '##le', 'ma', '##gda', 'line', '##tte', "'", 'ile', '(', 'w', '##ta', '64', '.', ')', '3', ':', '6', ',', '6', ':', '4', ',', '2', ':', '6', '.', '[SEP]']

```
```
NER model: [{'entity': 'B-PER', 'score': 0.99999887, 'index': 1, 'word': 'kai', 'start': None, 'end': None}, {'entity': 'B-PER', 'score': 0.97371966, 'index': 2, 'word': '##a', 'start': None, 'end': None}, {'entity': 'I-PER', 'score': 0.99999815, 'index': 3, 'word': 'kanepi', 'start': None, 'end': None}, {'entity': 'B-ORG', 'score': 0.63085276, 'index': 5, 'word': 'w', 'start': None, 'end': None}, {'entity': 'B-GPE', 'score': 0.99999934, 'index': 11, 'word': 'usa', 'start': None, 'end': None}, {'entity': 'B-GPE', 'score': 0.9999685, 'index': 14, 'word': 'cha', 'start': None, 'end': None}, {'entity': 'I-GPE', 'score': 0.8875574, 'index': 15, 'word': '##rl', 'start': None, 'end': None}, {'entity': 'I-GPE', 'score': 0.9996168, 'index': 16, 'word': '##est', 'start': None, 'end': None}, {'entity': 'I-GPE', 'score': 0.9992657, 'index': 17, 'word': '##onis', 'start': None, 'end': None}, {'entity': 'B-EVENT', 'score': 0.99999064, 'index': 19, 'word': 'w', 'start': None, 'end': None}, {'entity': 'I-EVENT', 'score': 0.9772493, 'index': 20, 'word': '##ta', 'start': None, 'end': None}, {'entity': 'I-EVENT', 'score': 0.99999076, 'index': 21, 'word': '500', 'start': None, 'end': None}, {'entity': 'I-EVENT', 'score': 0.99955636, 'index': 22, 'word': 'kategooria', 'start': None, 'end': None}, {'entity': 'B-TITLE', 'score': 0.8771319, 'index': 30, 'word': 'poola', 'start': None, 'end': None}, {'entity': 'B-PER', 'score': 0.99999785, 'index': 33, 'word': 'ma', 'start': None, 'end': None}, {'entity': 'B-PER', 'score': 0.9998398, 'index': 34, 'word': '##gda', 'start': None, 'end': None}, {'entity': 'I-PER', 'score': 0.9999987, 'index': 35, 'word': 'line', 'start': None, 'end': None}, {'entity': 'I-PER', 'score': 0.9999976, 'index': 36, 'word': '##tte', 'start': None, 'end': None}, {'entity': 'I-PER', 'score': 0.99999285, 'index': 37, 'word': "'", 'start': None, 'end': None}, {'entity': 'I-PER', 'score': 0.9999794, 'index': 38, 'word': 'ile', 'start': None, 'end': None}, {'entity': 'B-ORG', 'score': 0.7664479, 'index': 40, 'word': 'w', 'start': None, 'end': None}]
```

## Intended uses & limitations

This model can be used to find named entities from Estonian texts. The model is free to use for anyone. TartuNLP does not guarantee that the model is useful for anyone or anything. TartuNLP is not responsible for any results it generates.

## Training and evaluation data

The model was trained on two Estonian NER datasets:
- [The Reannotated Estonian NER corpus](https://metashare.ut.ee/repository/browse/reannotated-estonian-ner-corpus/bd43f1f614a511eca6e4fa163e9d45477d086613d2894fd5af79bf13e3f13594/)
- [The New Estonian NER corpus](https://metashare.ut.ee/repository/browse/new-estonian-ner-corpus/98b6706c963c11eba6e4fa163e9d45470bcd0533b6994c93ab8b8c628516ffed/)

Both datasets have been annotated with the same annotation scheme. For training this model, the datasets were joined.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 1024
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
- lr_scheduler_type: polynomial
- max num_epochs: 150
- early stopping limit: 20
- early stopping tol: 0.0001
- mixed_precision_training: Native AMP

### Training results

The final model was saved after epoch 53 (shown in bold) where the overall F1 was the highest on the development set.

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy | Date Precision | Date Recall | Date F1 | Date Number | Event Precision | Event Recall | Event F1 | Event Number | Gpe Precision | Gpe Recall | Gpe F1 | Gpe Number | Loc Precision | Loc Recall | Loc F1 | Loc Number | Money Precision | Money Recall | Money F1 | Money Number | Org Precision | Org Recall | Org F1 | Org Number | Per Precision | Per Recall | Per F1 | Per Number | Percent Precision | Percent Recall | Percent F1 | Percent Number | Prod Precision | Prod Recall | Prod F1 | Prod Number | Time Precision | Time Recall | Time F1 | Time Number | Title Precision | Title Recall | Title F1 | Title Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|:--------------:|:-----------:|:-------:|:-----------:|:---------------:|:------------:|:--------:|:------------:|:-------------:|:----------:|:------:|:----------:|:-------------:|:----------:|:------:|:----------:|:---------------:|:------------:|:--------:|:------------:|:-------------:|:----------:|:------:|:----------:|:-------------:|:----------:|:------:|:----------:|:-----------------:|:--------------:|:----------:|:--------------:|:--------------:|:-----------:|:-------:|:-----------:|:--------------:|:-----------:|:-------:|:-----------:|:---------------:|:------------:|:--------:|:------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.3252        | 1   | 1061  | 0.1628          | 0.6835    | 0.6083 | 0.6437 | 0.9526   | 0.5910         | 0.6022      | 0.5965  | 372         | 0.0             | 0.0          | 0.0      | 28           | 0.8073        | 0.7631     | 0.7846 | 840        | 0.1389        | 0.0336     | 0.0541 | 149        | 0.4217          | 0.3977       | 0.4094   | 88           | 0.5381        | 0.5280     | 0.5330 | 589        | 0.7917        | 0.8655     | 0.8270 | 751        | 0.6471            | 0.3014         | 0.4112     | 73             | 0.2581         | 0.0724      | 0.1131  | 221         | 0.1429         | 0.0889      | 0.1096  | 45          | 0.7805          | 0.6702       | 0.7211   | 191          | 0.6835            | 0.6083         | 0.6437     | 0.9526           |
| 0.1513        | 2   | 2122  | 0.1332          | 0.6906    | 0.7329 | 0.7111 | 0.9615   | 0.6185         | 0.7366      | 0.6724  | 372         | 0.0857          | 0.1071       | 0.0952   | 28           | 0.7874        | 0.8595     | 0.8219 | 840        | 0.4767        | 0.2752     | 0.3489 | 149        | 0.6848          | 0.7159       | 0.7000   | 88           | 0.6158        | 0.6231     | 0.6194 | 589        | 0.7770        | 0.9001     | 0.8341 | 751        | 0.9565            | 0.9041         | 0.9296     | 73             | 0.5            | 0.3620      | 0.4199  | 221         | 0.3571         | 0.3333      | 0.3448  | 45          | 0.6033          | 0.7644       | 0.6744   | 191          | 0.6906            | 0.7329         | 0.7111     | 0.9615           |
| 0.1131        | 3  | 3183  | 0.1281          | 0.7224    | 0.7338 | 0.7280 | 0.9638   | 0.7054         | 0.7339      | 0.7194  | 372         | 0.1053          | 0.1429       | 0.1212   | 28           | 0.8013        | 0.85       | 0.8250 | 840        | 0.5476        | 0.3087     | 0.3948 | 149        | 0.6386          | 0.6023       | 0.6199   | 88           | 0.6371        | 0.6469     | 0.6420 | 589        | 0.8235        | 0.8762     | 0.8490 | 751        | 0.9859            | 0.9589         | 0.9722     | 73             | 0.5148         | 0.3937      | 0.4462  | 221         | 0.5116         | 0.4889      | 0.5     | 45          | 0.6245          | 0.7749       | 0.6916   | 191          | 0.7224            | 0.7338         | 0.7280     | 0.9638           |
| 0.0884        | 4   | 4244  | 0.1354          | 0.7283    | 0.7386 | 0.7334 | 0.9639   | 0.6785         | 0.6694      | 0.6739  | 372         | 0.1795          | 0.25         | 0.2090   | 28           | 0.8231        | 0.8310     | 0.8270 | 840        | 0.6020        | 0.3960     | 0.4777 | 149        | 0.6092          | 0.6023       | 0.6057   | 88           | 0.6473        | 0.7012     | 0.6732 | 589        | 0.8351        | 0.8628     | 0.8487 | 751        | 1.0               | 0.9726         | 0.9861     | 73             | 0.5899         | 0.4751      | 0.5263  | 221         | 0.4524         | 0.4222      | 0.4368  | 45          | 0.6             | 0.7853       | 0.6803   | 191          | 0.7283            | 0.7386         | 0.7334     | 0.9639           |
| 0.0685        | 5   | 5305  | 0.1383          | 0.7224    | 0.7696 | 0.7453 | 0.9644   | 0.6635         | 0.7473      | 0.7029  | 372         | 0.26            | 0.4643       | 0.3333   | 28           | 0.8259        | 0.8357     | 0.8308 | 840        | 0.5913        | 0.4564     | 0.5152 | 149        | 0.6437          | 0.6364       | 0.64     | 88           | 0.6540        | 0.7284     | 0.6892 | 589        | 0.8070        | 0.8961     | 0.8492 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5693         | 0.5204      | 0.5437  | 221         | 0.5192         | 0.6         | 0.5567  | 45          | 0.6320          | 0.7644       | 0.6919   | 191          | 0.7224            | 0.7696         | 0.7453     | 0.9644           |
| 0.0532        | 6   | 6366  | 0.1493          | 0.7099    | 0.7613 | 0.7347 | 0.9631   | 0.6727         | 0.6962      | 0.6843  | 372         | 0.2308          | 0.5357       | 0.3226   | 28           | 0.8242        | 0.8262     | 0.8252 | 840        | 0.5877        | 0.4497     | 0.5095 | 149        | 0.6410          | 0.5682       | 0.6024   | 88           | 0.6232        | 0.7470     | 0.6795 | 589        | 0.8087        | 0.8895     | 0.8472 | 751        | 0.9672            | 0.8082         | 0.8806     | 73             | 0.5107         | 0.5385      | 0.5242  | 221         | 0.6190         | 0.5778      | 0.5977  | 45          | 0.6371          | 0.7906       | 0.7056   | 191          | 0.7099            | 0.7613         | 0.7347     | 0.9631           |
| 0.0403        | 7   | 7427  | 0.1592          | 0.7239    | 0.7592 | 0.7411 | 0.9642   | 0.6923         | 0.7016      | 0.6969  | 372         | 0.2857          | 0.5714       | 0.3810   | 28           | 0.8272        | 0.8262     | 0.8267 | 840        | 0.5752        | 0.4362     | 0.4962 | 149        | 0.6265          | 0.5909       | 0.6082   | 88           | 0.6402        | 0.6978     | 0.6677 | 589        | 0.8404        | 0.8762     | 0.8579 | 751        | 0.9859            | 0.9589         | 0.9722     | 73             | 0.5257         | 0.6018      | 0.5612  | 221         | 0.5870         | 0.6         | 0.5934  | 45          | 0.6235          | 0.8063       | 0.7032   | 191          | 0.7239            | 0.7592         | 0.7411     | 0.9642           |
| 0.0304        | 8   | 8488  | 0.1738          | 0.7301    | 0.7484 | 0.7392 | 0.9644   | 0.6866         | 0.6774      | 0.6820  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.8393        | 0.8083     | 0.8235 | 840        | 0.5882        | 0.4698     | 0.5224 | 149        | 0.6429          | 0.6136       | 0.6279   | 88           | 0.6608        | 0.6978     | 0.6788 | 589        | 0.8268        | 0.8708     | 0.8482 | 751        | 0.9595            | 0.9726         | 0.9660     | 73             | 0.5351         | 0.5520      | 0.5434  | 221         | 0.5208         | 0.5556      | 0.5376  | 45          | 0.6204          | 0.7958       | 0.6972   | 191          | 0.7301            | 0.7484         | 0.7392     | 0.9644           |
| 0.0234        | 9   | 9549  | 0.1860          | 0.7248    | 0.7625 | 0.7432 | 0.9641   | 0.6947         | 0.7097      | 0.7021  | 372         | 0.2963          | 0.5714       | 0.3902   | 28           | 0.8317        | 0.8298     | 0.8308 | 840        | 0.5913        | 0.4564     | 0.5152 | 149        | 0.6118          | 0.5909       | 0.6012   | 88           | 0.6361        | 0.7063     | 0.6693 | 589        | 0.8410        | 0.8735     | 0.8570 | 751        | 0.9859            | 0.9589         | 0.9722     | 73             | 0.5212         | 0.6109      | 0.5625  | 221         | 0.5417         | 0.5778      | 0.5591  | 45          | 0.6414          | 0.7958       | 0.7103   | 191          | 0.7248            | 0.7625         | 0.7432     | 0.9641           |
| 0.0178        | 10  | 10610 | 0.2037          | 0.7434    | 0.7383 | 0.7408 | 0.9640   | 0.7159         | 0.6774      | 0.6961  | 372         | 0.2857          | 0.4286       | 0.3429   | 28           | 0.8333        | 0.8333     | 0.8333 | 840        | 0.6262        | 0.4497     | 0.5234 | 149        | 0.6324          | 0.4886       | 0.5513   | 88           | 0.6568        | 0.6757     | 0.6661 | 589        | 0.8291        | 0.8722     | 0.8501 | 751        | 1.0               | 0.8219         | 0.9023     | 73             | 0.5672         | 0.5158      | 0.5403  | 221         | 0.5            | 0.5333      | 0.5161  | 45          | 0.6952          | 0.7644       | 0.7282   | 191          | 0.7434            | 0.7383         | 0.7408     | 0.9640           |
| 0.0147        | 11  | 11671 | 0.2114          | 0.7440    | 0.7233 | 0.7335 | 0.9643   | 0.7009         | 0.6613      | 0.6805  | 372         | 0.3030          | 0.3571       | 0.3279   | 28           | 0.8352        | 0.8024     | 0.8185 | 840        | 0.6238        | 0.4228     | 0.504  | 149        | 0.65            | 0.5909       | 0.6190   | 88           | 0.6436        | 0.6469     | 0.6452 | 589        | 0.8407        | 0.8575     | 0.8490 | 751        | 0.9315            | 0.9315         | 0.9315     | 73             | 0.5812         | 0.5023      | 0.5388  | 221         | 0.5476         | 0.5111      | 0.5287  | 45          | 0.6835          | 0.7801       | 0.7286   | 191          | 0.7440            | 0.7233         | 0.7335     | 0.9643           |
| 0.0118        | 12  | 12732 | 0.2218          | 0.7331    | 0.7532 | 0.7430 | 0.9649   | 0.7119         | 0.6909      | 0.7012  | 372         | 0.3488          | 0.5357       | 0.4225   | 28           | 0.8325        | 0.8405     | 0.8365 | 840        | 0.5303        | 0.4698     | 0.4982 | 149        | 0.65            | 0.5909       | 0.6190   | 88           | 0.6690        | 0.6587     | 0.6638 | 589        | 0.8178        | 0.8908     | 0.8528 | 751        | 0.9677            | 0.8219         | 0.8889     | 73             | 0.5408         | 0.5701      | 0.5551  | 221         | 0.5102         | 0.5556      | 0.5319  | 45          | 0.6567          | 0.8010       | 0.7217   | 191          | 0.7331            | 0.7532         | 0.7430     | 0.9649           |
| 0.0093        | 13  | 13793 | 0.2283          | 0.7495    | 0.7359 | 0.7427 | 0.9644   | 0.7163         | 0.6989      | 0.7075  | 372         | 0.3810          | 0.5714       | 0.4571   | 28           | 0.8612        | 0.7905     | 0.8243 | 840        | 0.6111        | 0.4430     | 0.5136 | 149        | 0.6145          | 0.5795       | 0.5965   | 88           | 0.6775        | 0.6740     | 0.6757 | 589        | 0.8346        | 0.8802     | 0.8568 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.5619         | 0.5339      | 0.5476  | 221         | 0.4            | 0.4889      | 0.4400  | 45          | 0.6812          | 0.7382       | 0.7085   | 191          | 0.7495            | 0.7359         | 0.7427     | 0.9644           |
| 0.0079        | 14  | 14854 | 0.2383          | 0.7371    | 0.7490 | 0.7430 | 0.9647   | 0.6727         | 0.7016      | 0.6868  | 372         | 0.3261          | 0.5357       | 0.4054   | 28           | 0.8453        | 0.8        | 0.8220 | 840        | 0.5963        | 0.4362     | 0.5039 | 149        | 0.625           | 0.5682       | 0.5952   | 88           | 0.6634        | 0.6927     | 0.6777 | 589        | 0.8433        | 0.8815     | 0.8620 | 751        | 0.9853            | 0.9178         | 0.9504     | 73             | 0.5427         | 0.5747      | 0.5582  | 221         | 0.5814         | 0.5556      | 0.5682  | 45          | 0.6513          | 0.8115       | 0.7226   | 191          | 0.7371            | 0.7490         | 0.7430     | 0.9647           |
| 0.0068        | 15  | 15915 | 0.2511          | 0.7255    | 0.7359 | 0.7306 | 0.9639   | 0.6826         | 0.6532      | 0.6676  | 372         | 0.3590          | 0.5          | 0.4179   | 28           | 0.8295        | 0.8167     | 0.8230 | 840        | 0.5263        | 0.4698     | 0.4965 | 149        | 0.6575          | 0.5455       | 0.5963   | 88           | 0.6549        | 0.6604     | 0.6577 | 589        | 0.8242        | 0.8802     | 0.8513 | 751        | 0.9833            | 0.8082         | 0.8872     | 73             | 0.5398         | 0.5520      | 0.5459  | 221         | 0.36           | 0.4         | 0.3789  | 45          | 0.6511          | 0.8010       | 0.7183   | 191          | 0.7255            | 0.7359         | 0.7306     | 0.9639           |
| 0.0061        | 16  | 16976 | 0.2497          | 0.7253    | 0.7690 | 0.7465 | 0.9648   | 0.6824         | 0.6989      | 0.6906  | 372         | 0.3333          | 0.5357       | 0.4110   | 28           | 0.8473        | 0.8321     | 0.8396 | 840        | 0.4583        | 0.5168     | 0.4858 | 149        | 0.6494          | 0.5682       | 0.6061   | 88           | 0.6556        | 0.7368     | 0.6938 | 589        | 0.8382        | 0.8828     | 0.8599 | 751        | 0.9841            | 0.8493         | 0.9118     | 73             | 0.5341         | 0.6380      | 0.5814  | 221         | 0.5            | 0.5333      | 0.5161  | 45          | 0.6622          | 0.7801       | 0.7163   | 191          | 0.7253            | 0.7690         | 0.7465     | 0.9648           |
| 0.0054        | 17  | 18037 | 0.2554          | 0.7323    | 0.7625 | 0.7471 | 0.9650   | 0.6870         | 0.6962      | 0.6916  | 372         | 0.3421          | 0.4643       | 0.3939   | 28           | 0.8463        | 0.8262     | 0.8361 | 840        | 0.5902        | 0.4832     | 0.5314 | 149        | 0.6753          | 0.5909       | 0.6303   | 88           | 0.6640        | 0.7148     | 0.6885 | 589        | 0.8317        | 0.8948     | 0.8621 | 751        | 0.9437            | 0.9178         | 0.9306     | 73             | 0.5210         | 0.5611      | 0.5403  | 221         | 0.5            | 0.5111      | 0.5055  | 45          | 0.6102          | 0.8115       | 0.6966   | 191          | 0.7323            | 0.7625         | 0.7471     | 0.9650           |
| 0.005         | 18  | 19098 | 0.2601          | 0.7273    | 0.7747 | 0.7503 | 0.9654   | 0.6970         | 0.7608      | 0.7275  | 372         | 0.2830          | 0.5357       | 0.3704   | 28           | 0.8320        | 0.8488     | 0.8403 | 840        | 0.5841        | 0.4430     | 0.5038 | 149        | 0.6477          | 0.6477       | 0.6477   | 88           | 0.6378        | 0.6995     | 0.6672 | 589        | 0.8501        | 0.8908     | 0.8700 | 751        | 0.9722            | 0.9589         | 0.9655     | 73             | 0.5323         | 0.5973      | 0.5629  | 221         | 0.4444         | 0.4444      | 0.4444  | 45          | 0.624           | 0.8168       | 0.7075   | 191          | 0.7273            | 0.7747         | 0.7503     | 0.9654           |
| 0.0044        | 19  | 20159 | 0.2602          | 0.7369    | 0.7616 | 0.7490 | 0.9656   | 0.7124         | 0.7124      | 0.7124  | 372         | 0.3415          | 0.5          | 0.4058   | 28           | 0.8239        | 0.8631     | 0.8430 | 840        | 0.6355        | 0.4564     | 0.5313 | 149        | 0.6667          | 0.6136       | 0.6391   | 88           | 0.6517        | 0.6638     | 0.6577 | 589        | 0.8405        | 0.8842     | 0.8618 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5144         | 0.5656      | 0.5388  | 221         | 0.5217         | 0.5333      | 0.5275  | 45          | 0.6550          | 0.7853       | 0.7143   | 191          | 0.7369            | 0.7616         | 0.7490     | 0.9656           |
| 0.004         | 20  | 21220 | 0.2677          | 0.7347    | 0.7702 | 0.7520 | 0.9658   | 0.7374         | 0.7097      | 0.7233  | 372         | 0.2857          | 0.4286       | 0.3429   | 28           | 0.8466        | 0.8345     | 0.8405 | 840        | 0.6050        | 0.4832     | 0.5373 | 149        | 0.6667          | 0.6136       | 0.6391   | 88           | 0.6593        | 0.7131     | 0.6852 | 589        | 0.8240        | 0.8975     | 0.8591 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.4981         | 0.5837      | 0.5375  | 221         | 0.5102         | 0.5556      | 0.5319  | 45          | 0.6371          | 0.8272       | 0.7198   | 191          | 0.7347            | 0.7702         | 0.7520     | 0.9658           |
| 0.0034        | 21  | 22281 | 0.2743          | 0.7386    | 0.7717 | 0.7548 | 0.9657   | 0.6984         | 0.7097      | 0.704   | 372         | 0.3784          | 0.5          | 0.4308   | 28           | 0.8475        | 0.8333     | 0.8403 | 840        | 0.6333        | 0.5101     | 0.5651 | 149        | 0.6190          | 0.5909       | 0.6047   | 88           | 0.6512        | 0.7385     | 0.6921 | 589        | 0.8428        | 0.8921     | 0.8668 | 751        | 0.9846            | 0.8767         | 0.9275     | 73             | 0.5513         | 0.5837      | 0.5670  | 221         | 0.5106         | 0.5333      | 0.5217  | 45          | 0.6379          | 0.8115       | 0.7143   | 191          | 0.7386            | 0.7717         | 0.7548     | 0.9657           |
| 0.0033        | 22  | 23342 | 0.2788          | 0.7418    | 0.7520 | 0.7469 | 0.9652   | 0.7143         | 0.6989      | 0.7065  | 372         | 0.3182          | 0.5          | 0.3889   | 28           | 0.8367        | 0.8298     | 0.8332 | 840        | 0.6168        | 0.4430     | 0.5156 | 149        | 0.6235          | 0.6023       | 0.6127   | 88           | 0.6758        | 0.6689     | 0.6724 | 589        | 0.8327        | 0.8815     | 0.8564 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5458         | 0.5928      | 0.5683  | 221         | 0.4783         | 0.4889      | 0.4835  | 45          | 0.6637          | 0.7853       | 0.7194   | 191          | 0.7418            | 0.7520         | 0.7469     | 0.9652           |
| 0.0033        | 23  | 24403 | 0.2831          | 0.7342    | 0.7535 | 0.7437 | 0.9650   | 0.6981         | 0.6962      | 0.6972  | 372         | 0.3784          | 0.5          | 0.4308   | 28           | 0.8499        | 0.8024     | 0.8255 | 840        | 0.5034        | 0.4966     | 0.5    | 149        | 0.6067          | 0.6136       | 0.6102   | 88           | 0.6581        | 0.6961     | 0.6766 | 589        | 0.8350        | 0.8961     | 0.8645 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5424         | 0.5792      | 0.5602  | 221         | 0.3774         | 0.4444      | 0.4082  | 45          | 0.7048          | 0.7749       | 0.7382   | 191          | 0.7342            | 0.7535         | 0.7437     | 0.9650           |
| 0.0029        | 24  | 25464 | 0.2931          | 0.7544    | 0.7380 | 0.7461 | 0.9648   | 0.7365         | 0.6989      | 0.7172  | 372         | 0.3590          | 0.5          | 0.4179   | 28           | 0.8535        | 0.7976     | 0.8246 | 840        | 0.5849        | 0.4161     | 0.4863 | 149        | 0.6622          | 0.5568       | 0.6049   | 88           | 0.6672        | 0.6706     | 0.6689 | 589        | 0.8474        | 0.8802     | 0.8635 | 751        | 0.9701            | 0.8904         | 0.9286     | 73             | 0.5550         | 0.5475      | 0.5513  | 221         | 0.4889         | 0.4889      | 0.4889  | 45          | 0.7023          | 0.7906       | 0.7438   | 191          | 0.7544            | 0.7380         | 0.7461     | 0.9648           |
| 0.0028        | 25  | 26525 | 0.2899          | 0.7489    | 0.7574 | 0.7531 | 0.9654   | 0.7021         | 0.7097      | 0.7059  | 372         | 0.3902          | 0.5714       | 0.4638   | 28           | 0.8635        | 0.8131     | 0.8375 | 840        | 0.6182        | 0.4564     | 0.5251 | 149        | 0.6471          | 0.625        | 0.6358   | 88           | 0.6613        | 0.6995     | 0.6799 | 589        | 0.8454        | 0.9028     | 0.8731 | 751        | 0.9583            | 0.9452         | 0.9517     | 73             | 0.5681         | 0.5475      | 0.5576  | 221         | 0.4222         | 0.4222      | 0.4222  | 45          | 0.6608          | 0.7853       | 0.7177   | 191          | 0.7489            | 0.7574         | 0.7531     | 0.9654           |
| 0.0023        | 26  | 27586 | 0.2922          | 0.7413    | 0.7532 | 0.7472 | 0.9649   | 0.6897         | 0.6989      | 0.6943  | 372         | 0.35            | 0.5          | 0.4118   | 28           | 0.85          | 0.8298     | 0.8398 | 840        | 0.6161        | 0.4631     | 0.5287 | 149        | 0.6486          | 0.5455       | 0.5926   | 88           | 0.6486        | 0.6927     | 0.6700 | 589        | 0.8457        | 0.8828     | 0.8638 | 751        | 0.9853            | 0.9178         | 0.9504     | 73             | 0.5636         | 0.5611      | 0.5624  | 221         | 0.3958         | 0.4222      | 0.4086  | 45          | 0.6638          | 0.7958       | 0.7238   | 191          | 0.7413            | 0.7532         | 0.7472     | 0.9649           |
| 0.0021        | 27  | 28647 | 0.2967          | 0.7514    | 0.7568 | 0.7541 | 0.9656   | 0.7081         | 0.7043      | 0.7062  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8547        | 0.8190     | 0.8365 | 840        | 0.5641        | 0.4430     | 0.4962 | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.6677        | 0.7097     | 0.6881 | 589        | 0.8459        | 0.8842     | 0.8646 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.5806         | 0.5701      | 0.5753  | 221         | 0.4898         | 0.5333      | 0.5106  | 45          | 0.7089          | 0.7906       | 0.7475   | 191          | 0.7514            | 0.7568         | 0.7541     | 0.9656           |
| 0.0025        | 28  | 29708 | 0.2957          | 0.7335    | 0.7622 | 0.7475 | 0.9651   | 0.7060         | 0.7231      | 0.7145  | 372         | 0.3077          | 0.4286       | 0.3582   | 28           | 0.8459        | 0.8429     | 0.8444 | 840        | 0.5069        | 0.4899     | 0.4983 | 149        | 0.6438          | 0.5341       | 0.5839   | 88           | 0.6838        | 0.7012     | 0.6924 | 589        | 0.8413        | 0.8895     | 0.8647 | 751        | 0.9552            | 0.8767         | 0.9143     | 73             | 0.4901         | 0.5611      | 0.5232  | 221         | 0.3818         | 0.4667      | 0.42    | 45          | 0.6580          | 0.7958       | 0.7204   | 191          | 0.7335            | 0.7622         | 0.7475     | 0.9651           |
| 0.0023        | 29  | 30769 | 0.3049          | 0.7455    | 0.7544 | 0.7499 | 0.9654   | 0.6997         | 0.7392      | 0.7190  | 372         | 0.3182          | 0.5          | 0.3889   | 28           | 0.8483        | 0.8119     | 0.8297 | 840        | 0.5630        | 0.5101     | 0.5352 | 149        | 0.6579          | 0.5682       | 0.6098   | 88           | 0.6791        | 0.7114     | 0.6949 | 589        | 0.8583        | 0.8628     | 0.8606 | 751        | 0.9853            | 0.9178         | 0.9504     | 73             | 0.5234         | 0.5566      | 0.5395  | 221         | 0.4565         | 0.4667      | 0.4615  | 45          | 0.7009          | 0.7853       | 0.7407   | 191          | 0.7455            | 0.7544         | 0.7499     | 0.9654           |
| 0.0018        | 30  | 31830 | 0.3042          | 0.7415    | 0.7679 | 0.7544 | 0.9654   | 0.6935         | 0.7419      | 0.7169  | 372         | 0.3333          | 0.5          | 0.4      | 28           | 0.8563        | 0.8226     | 0.8391 | 840        | 0.5878        | 0.5168     | 0.55   | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.6677        | 0.7470     | 0.7051 | 589        | 0.8544        | 0.8828     | 0.8684 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.5300         | 0.5204      | 0.5251  | 221         | 0.4375         | 0.4667      | 0.4516  | 45          | 0.6417          | 0.8063       | 0.7146   | 191          | 0.7415            | 0.7679         | 0.7544     | 0.9654           |
| 0.0017        | 31  | 32891 | 0.3071          | 0.7540    | 0.7481 | 0.7510 | 0.9660   | 0.7083         | 0.7312      | 0.7196  | 372         | 0.4054          | 0.5357       | 0.4615   | 28           | 0.8552        | 0.8226     | 0.8386 | 840        | 0.6311        | 0.4362     | 0.5159 | 149        | 0.6220          | 0.5795       | 0.6      | 88           | 0.6734        | 0.6757     | 0.6746 | 589        | 0.8626        | 0.8775     | 0.8700 | 751        | 0.9855            | 0.9315         | 0.9577     | 73             | 0.5307         | 0.5475      | 0.5390  | 221         | 0.3830         | 0.4         | 0.3913  | 45          | 0.7019          | 0.7644       | 0.7318   | 191          | 0.7540            | 0.7481         | 0.7510     | 0.9660           |
| 0.0018        | 32  | 33952 | 0.3190          | 0.7499    | 0.7553 | 0.7526 | 0.9656   | 0.7182         | 0.7124      | 0.7152  | 372         | 0.3333          | 0.5357       | 0.4110   | 28           | 0.8586        | 0.7952     | 0.8257 | 840        | 0.6116        | 0.4966     | 0.5481 | 149        | 0.6463          | 0.6023       | 0.6235   | 88           | 0.6805        | 0.6978     | 0.6890 | 589        | 0.8360        | 0.8895     | 0.8619 | 751        | 0.9855            | 0.9315         | 0.9577     | 73             | 0.5633         | 0.5837      | 0.5733  | 221         | 0.5106         | 0.5333      | 0.5217  | 45          | 0.6711          | 0.8010       | 0.7303   | 191          | 0.7499            | 0.7553         | 0.7526     | 0.9656           |
| 0.0018        | 33  | 35013 | 0.3094          | 0.7460    | 0.7774 | 0.7614 | 0.9665   | 0.7147         | 0.7473      | 0.7306  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8556        | 0.8393     | 0.8474 | 840        | 0.6273        | 0.4631     | 0.5328 | 149        | 0.6506          | 0.6136       | 0.6316   | 88           | 0.6787        | 0.7351     | 0.7058 | 589        | 0.8344        | 0.8988     | 0.8654 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5702         | 0.6063      | 0.5877  | 221         | 0.3036         | 0.3778      | 0.3366  | 45          | 0.6567          | 0.8010       | 0.7217   | 191          | 0.7460            | 0.7774         | 0.7614     | 0.9665           |
| 0.0015        | 34  | 36074 | 0.3091          | 0.7441    | 0.7759 | 0.7597 | 0.9665   | 0.7113         | 0.7285      | 0.7198  | 372         | 0.3404          | 0.5714       | 0.4267   | 28           | 0.8266        | 0.8512     | 0.8387 | 840        | 0.5405        | 0.5369     | 0.5387 | 149        | 0.6707          | 0.625        | 0.6471   | 88           | 0.6856        | 0.7182     | 0.7015 | 589        | 0.8517        | 0.8868     | 0.8689 | 751        | 1.0               | 0.9452         | 0.9718     | 73             | 0.5752         | 0.5882      | 0.5817  | 221         | 0.3878         | 0.4222      | 0.4043  | 45          | 0.6830          | 0.8010       | 0.7373   | 191          | 0.7441            | 0.7759         | 0.7597     | 0.9665           |
| 0.0015        | 35  | 37135 | 0.3185          | 0.7487    | 0.7619 | 0.7552 | 0.9660   | 0.6982         | 0.7339      | 0.7156  | 372         | 0.3415          | 0.5          | 0.4058   | 28           | 0.8685        | 0.8179     | 0.8424 | 840        | 0.5504        | 0.4765     | 0.5108 | 149        | 0.6353          | 0.6136       | 0.6243   | 88           | 0.6636        | 0.7267     | 0.6937 | 589        | 0.8654        | 0.8815     | 0.8734 | 751        | 1.0               | 0.9315         | 0.9645     | 73             | 0.55           | 0.5475      | 0.5488  | 221         | 0.3673         | 0.4         | 0.3830  | 45          | 0.6937          | 0.8063       | 0.7458   | 191          | 0.7487            | 0.7619         | 0.7552     | 0.9660           |
| 0.0015        | 36  | 38196 | 0.3203          | 0.7438    | 0.7649 | 0.7542 | 0.9660   | 0.6961         | 0.7204      | 0.7081  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8617        | 0.8381     | 0.8497 | 840        | 0.5203        | 0.5168     | 0.5185 | 149        | 0.6667          | 0.5909       | 0.6265   | 88           | 0.6710        | 0.7063     | 0.6882 | 589        | 0.8495        | 0.8868     | 0.8678 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.5561         | 0.5385      | 0.5471  | 221         | 0.42           | 0.4667      | 0.4421  | 45          | 0.6568          | 0.8115       | 0.7260   | 191          | 0.7438            | 0.7649         | 0.7542     | 0.9660           |
| 0.0013        | 37  | 39257 | 0.3298          | 0.7315    | 0.7732 | 0.7518 | 0.9656   | 0.6915         | 0.7231      | 0.7070  | 372         | 0.3333          | 0.5714       | 0.4211   | 28           | 0.8654        | 0.8190     | 0.8416 | 840        | 0.4793        | 0.5436     | 0.5094 | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.6656        | 0.7267     | 0.6948 | 589        | 0.8289        | 0.9028     | 0.8642 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5574         | 0.5928      | 0.5746  | 221         | 0.4043         | 0.4222      | 0.4130  | 45          | 0.6408          | 0.8220       | 0.7202   | 191          | 0.7315            | 0.7732         | 0.7518     | 0.9656           |
| 0.0012        | 38  | 40318 | 0.3311          | 0.7533    | 0.7610 | 0.7571 | 0.9664   | 0.7060         | 0.7231      | 0.7145  | 372         | 0.3571          | 0.5357       | 0.4286   | 28           | 0.8613        | 0.8357     | 0.8483 | 840        | 0.6339        | 0.4765     | 0.5441 | 149        | 0.6543          | 0.6023       | 0.6272   | 88           | 0.6528        | 0.7182     | 0.6839 | 589        | 0.8424        | 0.8828     | 0.8622 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.6031         | 0.5294      | 0.5639  | 221         | 0.4130         | 0.4222      | 0.4176  | 45          | 0.7122          | 0.7644       | 0.7374   | 191          | 0.7533            | 0.7610         | 0.7571     | 0.9664           |
| 0.0012        | 39  | 41379 | 0.3328          | 0.7444    | 0.7553 | 0.7498 | 0.9657   | 0.6818         | 0.7258      | 0.7031  | 372         | 0.3478          | 0.5714       | 0.4324   | 28           | 0.8561        | 0.8143     | 0.8347 | 840        | 0.6055        | 0.4430     | 0.5116 | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.6715        | 0.7046     | 0.6877 | 589        | 0.8461        | 0.8708     | 0.8583 | 751        | 0.9706            | 0.9041         | 0.9362     | 73             | 0.5665         | 0.5973      | 0.5815  | 221         | 0.4082         | 0.4444      | 0.4255  | 45          | 0.6770          | 0.8010       | 0.7338   | 191          | 0.7444            | 0.7553         | 0.7498     | 0.9657           |
| 0.0014        | 40  | 42440 | 0.3415          | 0.7421    | 0.7437 | 0.7429 | 0.9641   | 0.6931         | 0.7043      | 0.6987  | 372         | 0.3488          | 0.5357       | 0.4225   | 28           | 0.8422        | 0.8262     | 0.8341 | 840        | 0.6190        | 0.4362     | 0.5118 | 149        | 0.6622          | 0.5568       | 0.6049   | 88           | 0.6888        | 0.6350     | 0.6608 | 589        | 0.8175        | 0.8828     | 0.8489 | 751        | 1.0               | 0.9178         | 0.9571     | 73             | 0.5584         | 0.5837      | 0.5708  | 221         | 0.4043         | 0.4222      | 0.4130  | 45          | 0.6580          | 0.7958       | 0.7204   | 191          | 0.7421            | 0.7437         | 0.7429     | 0.9641           |
| 0.0013        | 41  | 43501 | 0.3401          | 0.7501    | 0.7487 | 0.7494 | 0.9651   | 0.6915         | 0.7231      | 0.7070  | 372         | 0.3421          | 0.4643       | 0.3939   | 28           | 0.8545        | 0.8179     | 0.8358 | 840        | 0.6346        | 0.4430     | 0.5217 | 149        | 0.6812          | 0.5341       | 0.5987   | 88           | 0.6728        | 0.6808     | 0.6768 | 589        | 0.8380        | 0.8748     | 0.8560 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.5860         | 0.5701      | 0.5780  | 221         | 0.4423         | 0.5111      | 0.4742  | 45          | 0.6787          | 0.7853       | 0.7282   | 191          | 0.7501            | 0.7487         | 0.7494     | 0.9651           |
| 0.0011        | 42  | 44562 | 0.3468          | 0.7426    | 0.7687 | 0.7554 | 0.9650   | 0.6965         | 0.7527      | 0.7235  | 372         | 0.3488          | 0.5357       | 0.4225   | 28           | 0.8667        | 0.8202     | 0.8428 | 840        | 0.6408        | 0.4430     | 0.5238 | 149        | 0.6709          | 0.6023       | 0.6347   | 88           | 0.6902        | 0.7148     | 0.7023 | 589        | 0.8404        | 0.8975     | 0.8680 | 751        | 0.9444            | 0.9315         | 0.9379     | 73             | 0.5191         | 0.6154      | 0.5631  | 221         | 0.3469         | 0.3778      | 0.3617  | 45          | 0.6210          | 0.8063       | 0.7016   | 191          | 0.7426            | 0.7687         | 0.7554     | 0.9650           |
| 0.0015        | 43  | 45623 | 0.3440          | 0.7566    | 0.7422 | 0.7493 | 0.9648   | 0.6937         | 0.7366      | 0.7145  | 372         | 0.3846          | 0.5357       | 0.4478   | 28           | 0.8608        | 0.8095     | 0.8344 | 840        | 0.6082        | 0.3960     | 0.4797 | 149        | 0.7             | 0.5568       | 0.6203   | 88           | 0.6766        | 0.6570     | 0.6667 | 589        | 0.8317        | 0.8881     | 0.8590 | 751        | 0.9701            | 0.8904         | 0.9286     | 73             | 0.6224         | 0.5520      | 0.5851  | 221         | 0.3913         | 0.4         | 0.3956  | 45          | 0.7081          | 0.7749       | 0.74     | 191          | 0.7566            | 0.7422         | 0.7493     | 0.9648           |
| 0.0011        | 44  | 46684 | 0.3354          | 0.7565    | 0.7640 | 0.7602 | 0.9664   | 0.7062         | 0.7366      | 0.7211  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8483        | 0.8452     | 0.8468 | 840        | 0.6095        | 0.4295     | 0.5039 | 149        | 0.6883          | 0.6023       | 0.6424   | 88           | 0.6880        | 0.6740     | 0.6810 | 589        | 0.8517        | 0.8948     | 0.8727 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.6238         | 0.5928      | 0.6079  | 221         | 0.3830         | 0.4         | 0.3913  | 45          | 0.65            | 0.8168       | 0.7239   | 191          | 0.7565            | 0.7640         | 0.7602     | 0.9664           |
| 0.0011        | 45  | 47745 | 0.3347          | 0.7485    | 0.7622 | 0.7553 | 0.9655   | 0.7088         | 0.7392      | 0.7237  | 372         | 0.3636          | 0.5714       | 0.4444   | 28           | 0.8603        | 0.8286     | 0.8441 | 840        | 0.5882        | 0.4698     | 0.5224 | 149        | 0.6023          | 0.6023       | 0.6023   | 88           | 0.6770        | 0.6689     | 0.6729 | 589        | 0.8417        | 0.8921     | 0.8662 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.6037         | 0.5928      | 0.5982  | 221         | 0.4583         | 0.4889      | 0.4731  | 45          | 0.6275          | 0.8115       | 0.7078   | 191          | 0.7485            | 0.7622         | 0.7553     | 0.9655           |
| 0.0011        | 46  | 48806 | 0.3421          | 0.7481    | 0.7640 | 0.7559 | 0.9657   | 0.7261         | 0.7339      | 0.7299  | 372         | 0.3171          | 0.4643       | 0.3768   | 28           | 0.8570        | 0.8202     | 0.8382 | 840        | 0.5691        | 0.4698     | 0.5147 | 149        | 0.6429          | 0.6136       | 0.6279   | 88           | 0.6769        | 0.7114     | 0.6937 | 589        | 0.8311        | 0.8908     | 0.8599 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5714         | 0.5611      | 0.5662  | 221         | 0.5            | 0.5556      | 0.5263  | 45          | 0.6638          | 0.7958       | 0.7238   | 191          | 0.7481            | 0.7640         | 0.7559     | 0.9657           |
| 0.0009        | 47  | 49867 | 0.3487          | 0.7496    | 0.7604 | 0.7550 | 0.9656   | 0.7158         | 0.7043      | 0.7100  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.86          | 0.8190     | 0.8390 | 840        | 0.5496        | 0.4832     | 0.5143 | 149        | 0.7162          | 0.6023       | 0.6543   | 88           | 0.6745        | 0.7284     | 0.7004 | 589        | 0.8346        | 0.8802     | 0.8568 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5566         | 0.5339      | 0.5450  | 221         | 0.5349         | 0.5111      | 0.5227  | 45          | 0.6828          | 0.8115       | 0.7416   | 191          | 0.7496            | 0.7604         | 0.7550     | 0.9656           |
| 0.0009        | 48  | 50928 | 0.3470          | 0.7414    | 0.7649 | 0.7529 | 0.9651   | 0.7092         | 0.7473      | 0.7277  | 372         | 0.3333          | 0.5357       | 0.4110   | 28           | 0.8541        | 0.8226     | 0.8381 | 840        | 0.5847        | 0.4631     | 0.5169 | 149        | 0.6835          | 0.6136       | 0.6467   | 88           | 0.6801        | 0.7148     | 0.6970 | 589        | 0.8319        | 0.8895     | 0.8597 | 751        | 0.9571            | 0.9178         | 0.9371     | 73             | 0.5307         | 0.5475      | 0.5390  | 221         | 0.4583         | 0.4889      | 0.4731  | 45          | 0.6364          | 0.8063       | 0.7113   | 191          | 0.7414            | 0.7649         | 0.7529     | 0.9651           |
| 0.0011        | 49  | 51989 | 0.3389          | 0.7435    | 0.7664 | 0.7547 | 0.9659   | 0.6957         | 0.7312      | 0.7130  | 372         | 0.3590          | 0.5          | 0.4179   | 28           | 0.8561        | 0.8286     | 0.8421 | 840        | 0.6636        | 0.4899     | 0.5637 | 149        | 0.6136          | 0.6136       | 0.6136   | 88           | 0.6732        | 0.6995     | 0.6861 | 589        | 0.8251        | 0.8921     | 0.8573 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5746         | 0.5928      | 0.5835  | 221         | 0.4348         | 0.4444      | 0.4396  | 45          | 0.6390          | 0.8063       | 0.7130   | 191          | 0.7435            | 0.7664         | 0.7547     | 0.9659           |
| 0.0009        | 50  | 53050 | 0.3557          | 0.7490    | 0.7640 | 0.7564 | 0.9659   | 0.6948         | 0.6855      | 0.6901  | 372         | 0.3947          | 0.5357       | 0.4545   | 28           | 0.8584        | 0.8298     | 0.8438 | 840        | 0.6455        | 0.4765     | 0.5483 | 149        | 0.6933          | 0.5909       | 0.6380   | 88           | 0.6745        | 0.7317     | 0.7020 | 589        | 0.8296        | 0.8948     | 0.8610 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.6082         | 0.5339      | 0.5687  | 221         | 0.4043         | 0.4222      | 0.4130  | 45          | 0.6270          | 0.8272       | 0.7133   | 191          | 0.7490            | 0.7640         | 0.7564     | 0.9659           |
| 0.0008        | 51  | 54111 | 0.3492          | 0.7516    | 0.7601 | 0.7558 | 0.9662   | 0.7104         | 0.6989      | 0.7046  | 372         | 0.3714          | 0.4643       | 0.4127   | 28           | 0.8545        | 0.8321     | 0.8432 | 840        | 0.6496        | 0.5101     | 0.5714 | 149        | 0.625           | 0.5682       | 0.5952   | 88           | 0.6722        | 0.6893     | 0.6806 | 589        | 0.8413        | 0.8895     | 0.8647 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5611         | 0.5611      | 0.5611  | 221         | 0.4792         | 0.5111      | 0.4946  | 45          | 0.6724          | 0.8168       | 0.7376   | 191          | 0.7516            | 0.7601         | 0.7558     | 0.9662           |
| 0.0008        | 52  | 55172 | 0.3432          | 0.7526    | 0.7625 | 0.7575 | 0.9661   | 0.7044         | 0.7366      | 0.7201  | 372         | 0.3571          | 0.5357       | 0.4286   | 28           | 0.8610        | 0.8262     | 0.8433 | 840        | 0.6140        | 0.4698     | 0.5323 | 149        | 0.6667          | 0.5909       | 0.6265   | 88           | 0.6766        | 0.6927     | 0.6846 | 589        | 0.8403        | 0.8895     | 0.8642 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5849         | 0.5611      | 0.5727  | 221         | 0.46           | 0.5111      | 0.4842  | 45          | 0.6681          | 0.8115       | 0.7329   | 191          | 0.7526            | 0.7625         | 0.7575     | 0.9661           |
| **0.0006**        | **53**  | **56233** | **0.3565**          | **0.7615**   | **0.7747** | **0.7681** | **0.9672**  | **0.7305**     | **0.7285**    | **0.7295** | **372**       | **0.3721**          | **0.5714**       | **0.4507**   | **28**           | **0.8679**        | **0.8369**     | **0.8521** | **840**        | **0.6545**        | **0.4832**     | **0.5560** | **149**        | **0.6625**          | **0.6023**       | **0.6310**   | **88**           | **0.6761**        | **0.7267**     | **0.7005** | **589**        | **0.8255**        | **0.9068**     | **0.8642** | **751**        | **1.0**               | **0.9589**         | **0.9790**     | **73**             | **0.6030**         | **0.5430**      | **0.5714**  | **221**         | **0.5682**         | **0.5556**      | **0.5618**  | **45**          | **0.7**             | **0.8063**       | **0.7494**   | **191**          | **0.7615**            | **0.7747**         | **0.7681**     | **0.9672**           |
| 0.0008        | 54  | 57294 | 0.3480          | 0.7590    | 0.7631 | 0.7610 | 0.9668   | 0.7452         | 0.7312      | 0.7381  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.8589        | 0.8190     | 0.8385 | 840        | 0.5935        | 0.4899     | 0.5368 | 149        | 0.7027          | 0.5909       | 0.6420   | 88           | 0.6924        | 0.6842     | 0.6883 | 589        | 0.8432        | 0.8948     | 0.8682 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5856         | 0.5882      | 0.5869  | 221         | 0.5102         | 0.5556      | 0.5319  | 45          | 0.6513          | 0.8115       | 0.7226   | 191          | 0.7590            | 0.7631         | 0.7610     | 0.9668           |
| 0.0008        | 55  | 58355 | 0.3568          | 0.7601    | 0.7622 | 0.7612 | 0.9663   | 0.7228         | 0.7151      | 0.7189  | 372         | 0.3571          | 0.5357       | 0.4286   | 28           | 0.8429        | 0.8429     | 0.8429 | 840        | 0.6634        | 0.4497     | 0.536  | 149        | 0.7             | 0.5568       | 0.6203   | 88           | 0.6828        | 0.7165     | 0.6993 | 589        | 0.8655        | 0.8828     | 0.8741 | 751        | 0.9853            | 0.9178         | 0.9504     | 73             | 0.5909         | 0.5294      | 0.5585  | 221         | 0.5106         | 0.5333      | 0.5217  | 45          | 0.6429          | 0.8010       | 0.7133   | 191          | 0.7601            | 0.7622         | 0.7612     | 0.9663           |
| 0.0009        | 56  | 59416 | 0.3498          | 0.7542    | 0.7580 | 0.7561 | 0.9661   | 0.7178         | 0.7043      | 0.7110  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.8379        | 0.8429     | 0.8404 | 840        | 0.6634        | 0.4497     | 0.536  | 149        | 0.6322          | 0.625        | 0.6286   | 88           | 0.6895        | 0.6825     | 0.6860 | 589        | 0.8513        | 0.8842     | 0.8674 | 751        | 0.9577            | 0.9315         | 0.9444     | 73             | 0.5613         | 0.5385      | 0.5497  | 221         | 0.5111         | 0.5111      | 0.5111  | 45          | 0.6667          | 0.8063       | 0.7299   | 191          | 0.7542            | 0.7580         | 0.7561     | 0.9661           |
| 0.0007        | 57  | 60477 | 0.3486          | 0.7479    | 0.7711 | 0.7593 | 0.9663   | 0.7143         | 0.7392      | 0.7266  | 372         | 0.3571          | 0.5357       | 0.4286   | 28           | 0.8417        | 0.8417     | 0.8417 | 840        | 0.5923        | 0.5168     | 0.5520 | 149        | 0.6667          | 0.6136       | 0.6391   | 88           | 0.6720        | 0.7165     | 0.6935 | 589        | 0.8562        | 0.8802     | 0.8680 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5670         | 0.5747      | 0.5708  | 221         | 0.4583         | 0.4889      | 0.4731  | 45          | 0.6623          | 0.8010       | 0.7251   | 191          | 0.7479            | 0.7711         | 0.7593     | 0.9663           |
| 0.0007        | 58  | 61538 | 0.3497          | 0.7539    | 0.7744 | 0.7640 | 0.9667   | 0.7143         | 0.7392      | 0.7266  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8449        | 0.8429     | 0.8439 | 840        | 0.6429        | 0.4832     | 0.5517 | 149        | 0.6667          | 0.5909       | 0.6265   | 88           | 0.6708        | 0.7267     | 0.6976 | 589        | 0.8499        | 0.8975     | 0.8731 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.6108         | 0.5611      | 0.5849  | 221         | 0.5            | 0.4889      | 0.4944  | 45          | 0.6525          | 0.8063       | 0.7213   | 191          | 0.7539            | 0.7744         | 0.7640     | 0.9667           |
| 0.0008        | 59  | 62599 | 0.3581          | 0.7474    | 0.7762 | 0.7615 | 0.9662   | 0.7183         | 0.7473      | 0.7325  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.8439        | 0.8429     | 0.8434 | 840        | 0.5467        | 0.5503     | 0.5485 | 149        | 0.6709          | 0.6023       | 0.6347   | 88           | 0.6693        | 0.7250     | 0.6960 | 589        | 0.8454        | 0.8881     | 0.8662 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5961         | 0.5475      | 0.5708  | 221         | 0.5            | 0.5333      | 0.5161  | 45          | 0.6769          | 0.8115       | 0.7381   | 191          | 0.7474            | 0.7762         | 0.7615     | 0.9662           |
| 0.0007        | 60  | 63660 | 0.3636          | 0.7494    | 0.7676 | 0.7584 | 0.9662   | 0.7016         | 0.7204      | 0.7109  | 372         | 0.3488          | 0.5357       | 0.4225   | 28           | 0.8489        | 0.8357     | 0.8422 | 840        | 0.6           | 0.4832     | 0.5353 | 149        | 0.6538          | 0.5795       | 0.6145   | 88           | 0.6828        | 0.7199     | 0.7008 | 589        | 0.8476        | 0.8815     | 0.8642 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5579         | 0.5882      | 0.5727  | 221         | 0.4762         | 0.4444      | 0.4598  | 45          | 0.6797          | 0.8220       | 0.7441   | 191          | 0.7494            | 0.7676         | 0.7584     | 0.9662           |
| 0.0008        | 61  | 64721 | 0.3646          | 0.7538    | 0.7574 | 0.7556 | 0.9660   | 0.6854         | 0.7204      | 0.7025  | 372         | 0.3659          | 0.5357       | 0.4348   | 28           | 0.8573        | 0.8369     | 0.8470 | 840        | 0.6306        | 0.4698     | 0.5385 | 149        | 0.6667          | 0.5909       | 0.6265   | 88           | 0.6896        | 0.6978     | 0.6937 | 589        | 0.8495        | 0.8722     | 0.8607 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5728         | 0.5520      | 0.5622  | 221         | 0.375          | 0.4         | 0.3871  | 45          | 0.6830          | 0.8010       | 0.7373   | 191          | 0.7538            | 0.7574         | 0.7556     | 0.9660           |
| 0.0006        | 62  | 65782 | 0.3697          | 0.7510    | 0.7460 | 0.7485 | 0.9651   | 0.6885         | 0.7070      | 0.6976  | 372         | 0.4286          | 0.5357       | 0.4762   | 28           | 0.8663        | 0.7869     | 0.8247 | 840        | 0.5902        | 0.4832     | 0.5314 | 149        | 0.6757          | 0.5682       | 0.6173   | 88           | 0.6667        | 0.6927     | 0.6794 | 589        | 0.8432        | 0.8948     | 0.8682 | 751        | 0.9851            | 0.9041         | 0.9429     | 73             | 0.5829         | 0.5566      | 0.5694  | 221         | 0.3673         | 0.4         | 0.3830  | 45          | 0.6995          | 0.7801       | 0.7376   | 191          | 0.7510            | 0.7460         | 0.7485     | 0.9651           |
| 0.0006        | 63  | 66843 | 0.3661          | 0.7504    | 0.7502 | 0.7503 | 0.9655   | 0.6909         | 0.6909      | 0.6909  | 372         | 0.4286          | 0.5357       | 0.4762   | 28           | 0.8571        | 0.8143     | 0.8352 | 840        | 0.5814        | 0.5034     | 0.5396 | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.7013        | 0.6655     | 0.6829 | 589        | 0.8348        | 0.8948     | 0.8638 | 751        | 0.9571            | 0.9178         | 0.9371     | 73             | 0.5570         | 0.5747      | 0.5657  | 221         | 0.3830         | 0.4         | 0.3913  | 45          | 0.6786          | 0.7958       | 0.7325   | 191          | 0.7504            | 0.7502         | 0.7503     | 0.9655           |
| 0.0006        | 64  | 67904 | 0.3711          | 0.7404    | 0.7628 | 0.7514 | 0.9656   | 0.6911         | 0.7097      | 0.7003  | 372         | 0.3784          | 0.5          | 0.4308   | 28           | 0.8455        | 0.8405     | 0.8430 | 840        | 0.6           | 0.5034     | 0.5474 | 149        | 0.65            | 0.5909       | 0.6190   | 88           | 0.6667        | 0.7029     | 0.6843 | 589        | 0.8350        | 0.8961     | 0.8645 | 751        | 0.9714            | 0.9315         | 0.9510     | 73             | 0.5673         | 0.5339      | 0.5501  | 221         | 0.2917         | 0.3111      | 0.3011  | 45          | 0.6568          | 0.8115       | 0.7260   | 191          | 0.7404            | 0.7628         | 0.7514     | 0.9656           |
| 0.0007        | 65  | 68965 | 0.3672          | 0.7377    | 0.7696 | 0.7533 | 0.9661   | 0.7005         | 0.7419      | 0.7206  | 372         | 0.3333          | 0.5357       | 0.4110   | 28           | 0.8433        | 0.8393     | 0.8413 | 840        | 0.5839        | 0.5369     | 0.5594 | 149        | 0.6506          | 0.6136       | 0.6316   | 88           | 0.6840        | 0.7131     | 0.6983 | 589        | 0.8412        | 0.8815     | 0.8609 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.5427         | 0.5747      | 0.5582  | 221         | 0.3019         | 0.3556      | 0.3265  | 45          | 0.6360          | 0.7958       | 0.7070   | 191          | 0.7377            | 0.7696         | 0.7533     | 0.9661           |
| 0.0005        | 66  | 70026 | 0.3768          | 0.7496    | 0.7520 | 0.7508 | 0.9657   | 0.6903         | 0.7070      | 0.6985  | 372         | 0.3415          | 0.5          | 0.4058   | 28           | 0.8454        | 0.8333     | 0.8393 | 840        | 0.6372        | 0.4832     | 0.5496 | 149        | 0.6795          | 0.6023       | 0.6386   | 88           | 0.6914        | 0.6655     | 0.6782 | 589        | 0.8483        | 0.8788     | 0.8633 | 751        | 0.9577            | 0.9315         | 0.9444     | 73             | 0.5714         | 0.5792      | 0.5753  | 221         | 0.3            | 0.3333      | 0.3158  | 45          | 0.6696          | 0.7958       | 0.7273   | 191          | 0.7496            | 0.7520         | 0.7508     | 0.9657           |
| 0.0007        | 67  | 71087 | 0.3682          | 0.7461    | 0.7664 | 0.7561 | 0.9656   | 0.7094         | 0.7285      | 0.7188  | 372         | 0.3409          | 0.5357       | 0.4167   | 28           | 0.8563        | 0.8369     | 0.8465 | 840        | 0.6290        | 0.5235     | 0.5714 | 149        | 0.6974          | 0.6023       | 0.6463   | 88           | 0.6935        | 0.6876     | 0.6905 | 589        | 0.8363        | 0.8842     | 0.8595 | 751        | 0.9437            | 0.9178         | 0.9306     | 73             | 0.5175         | 0.6018      | 0.5565  | 221         | 0.4694         | 0.5111      | 0.4894  | 45          | 0.6483          | 0.8010       | 0.7166   | 191          | 0.7461            | 0.7664         | 0.7561     | 0.9656           |
| 0.0005        | 68  | 72148 | 0.3815          | 0.7590    | 0.7416 | 0.7502 | 0.9654   | 0.7092         | 0.7016      | 0.7054  | 372         | 0.4054          | 0.5357       | 0.4615   | 28           | 0.8489        | 0.8095     | 0.8288 | 840        | 0.6796        | 0.4698     | 0.5556 | 149        | 0.6456          | 0.5795       | 0.6108   | 88           | 0.6801        | 0.6570     | 0.6684 | 589        | 0.8476        | 0.8815     | 0.8642 | 751        | 0.9571            | 0.9178         | 0.9371     | 73             | 0.615          | 0.5566      | 0.5843  | 221         | 0.4348         | 0.4444      | 0.4396  | 45          | 0.6759          | 0.7644       | 0.7174   | 191          | 0.7590            | 0.7416         | 0.7502     | 0.9654           |
| 0.0006        | 69  | 73209 | 0.3919          | 0.7494    | 0.7487 | 0.7491 | 0.9650   | 0.6888         | 0.6962      | 0.6925  | 372         | 0.3590          | 0.5          | 0.4179   | 28           | 0.8416        | 0.8095     | 0.8252 | 840        | 0.5865        | 0.5235     | 0.5532 | 149        | 0.6901          | 0.5568       | 0.6164   | 88           | 0.6950        | 0.6808     | 0.6878 | 589        | 0.8490        | 0.8908     | 0.8694 | 751        | 1.0               | 0.9041         | 0.9496     | 73             | 0.5662         | 0.5611      | 0.5636  | 221         | 0.3265         | 0.3556      | 0.3404  | 45          | 0.6881          | 0.7853       | 0.7335   | 191          | 0.7494            | 0.7487         | 0.7491     | 0.9650           |
| 0.0006        | 70  | 74270 | 0.3704          | 0.7587    | 0.7619 | 0.7603 | 0.9666   | 0.6891         | 0.7151      | 0.7018  | 372         | 0.3947          | 0.5357       | 0.4545   | 28           | 0.8376        | 0.8536     | 0.8455 | 840        | 0.6697        | 0.4899     | 0.5659 | 149        | 0.6420          | 0.5909       | 0.6154   | 88           | 0.7018        | 0.6791     | 0.6903 | 589        | 0.8491        | 0.8842     | 0.8663 | 751        | 0.9857            | 0.9452         | 0.9650     | 73             | 0.6219         | 0.5656      | 0.5924  | 221         | 0.3913         | 0.4         | 0.3956  | 45          | 0.6802          | 0.7906       | 0.7312   | 191          | 0.7587            | 0.7619         | 0.7603     | 0.9666           |
| 0.0005        | 71  | 75331 | 0.3841          | 0.7501    | 0.7634 | 0.7567 | 0.9659   | 0.7005         | 0.6855      | 0.6929  | 372         | 0.4054          | 0.5357       | 0.4615   | 28           | 0.8531        | 0.8298     | 0.8413 | 840        | 0.6293        | 0.4899     | 0.5509 | 149        | 0.6410          | 0.5682       | 0.6024   | 88           | 0.6774        | 0.7165     | 0.6964 | 589        | 0.8264        | 0.9001     | 0.8617 | 751        | 0.9706            | 0.9041         | 0.9362     | 73             | 0.5882         | 0.5882      | 0.5882  | 221         | 0.4545         | 0.4444      | 0.4494  | 45          | 0.6864          | 0.7906       | 0.7348   | 191          | 0.7501            | 0.7634         | 0.7567     | 0.9659           |
| 0.0005        | 72  | 76392 | 0.3830          | 0.7605    | 0.7496 | 0.7550 | 0.9655   | 0.7036         | 0.6828      | 0.6930  | 372         | 0.3824          | 0.4643       | 0.4194   | 28           | 0.8618        | 0.8238     | 0.8424 | 840        | 0.6542        | 0.4698     | 0.5469 | 149        | 0.6582          | 0.5909       | 0.6228   | 88           | 0.6935        | 0.6723     | 0.6828 | 589        | 0.8476        | 0.8815     | 0.8642 | 751        | 0.9577            | 0.9315         | 0.9444     | 73             | 0.5830         | 0.5882      | 0.5856  | 221         | 0.4043         | 0.4222      | 0.4130  | 45          | 0.6892          | 0.8010       | 0.7409   | 191          | 0.7605            | 0.7496         | 0.7550     | 0.9655           |
| 0.0006        | 73  | 77453 | 0.3839          | 0.7611    | 0.7547 | 0.7579 | 0.9661   | 0.712          | 0.7177      | 0.7149  | 372         | 0.3429          | 0.4286       | 0.3810   | 28           | 0.8494        | 0.8393     | 0.8443 | 840        | 0.6542        | 0.4698     | 0.5469 | 149        | 0.6538          | 0.5795       | 0.6145   | 88           | 0.6877        | 0.6655     | 0.6764 | 589        | 0.8428        | 0.8921     | 0.8668 | 751        | 0.9710            | 0.9178         | 0.9437     | 73             | 0.6257         | 0.5294      | 0.5735  | 221         | 0.4468         | 0.4667      | 0.4565  | 45          | 0.6814          | 0.8063       | 0.7386   | 191          | 0.7611            | 0.7547         | 0.7579     | 0.9661           |


### Framework versions

- Transformers 4.16.2
- Pytorch 1.10.2+cu113
- Datasets 1.18.3
- Tokenizers 0.11.0

{"language": "et", "license": "cc-by-4.0", "widget": [{"text": "Eesti President on Alar Karis."}], "base_model": "tartuNLP/EstBERT"}
transformers pytorch safetensors bert token-classification et base_model:tartuNLP/EstBERT license:cc-by-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #safetensors #bert #token-classification #et #base_model-tartuNLP/EstBERT #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 

Estonian NER model based on EstBERT
===================================


This model is a fine-tuned version of tartuNLP/EstBERT on the Estonian NER dataset. The model was trained by tartuNLP, the NLP research group at the institute of Computer Science at the University of Tartu.


It achieves the following results on the test set:


* Loss: 0.3565
* Precision: 0.7612
* Recall: 0.7744
* F1: 0.7678
* Accuracy: 0.9672


The entity-level results are as follows:



How to use
----------


You can use this model with Transformers pipeline for NER. Post-processing of results may be necessary as the model occasionally tags subword tokens as entities.


Intended uses & limitations
---------------------------


This model can be used to find named entities from Estonian texts. The model is free to use for anyone. TartuNLP does not guarantee that the model is useful for anyone or anything. TartuNLP is not responsible for any results it generates.


Training and evaluation data
----------------------------


The model was trained on two Estonian NER datasets:


* The Reannotated Estonian NER corpus
* The New Estonian NER corpus


Both datasets have been annotated with the same annotation scheme. For training this model, the datasets were joined.


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 1e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 1024
* optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
* lr\_scheduler\_type: polynomial
* max num\_epochs: 150
* early stopping limit: 20
* early stopping tol: 0.0001
* mixed\_precision\_training: Native AMP


### Training results


The final model was saved after epoch 53 (shown in bold) where the overall F1 was the highest on the development set.



### Framework versions


* Transformers 4.16.2
* Pytorch 1.10.2+cu113
* Datasets 1.18.3
* Tokenizers 0.11.0



### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 1e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 1024
* optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
* lr\_scheduler\_type: polynomial
* max num\_epochs: 150
* early stopping limit: 20
* early stopping tol: 0.0001
* mixed\_precision\_training: Native AMP ### Training results


The final model was saved after epoch 53 (shown in bold) where the overall F1 was the highest on the development set. ### Framework versions


* Transformers 4.16.2
* Pytorch 1.10.2+cu113
* Datasets 1.18.3
* Tokenizers 0.11.0
TAGS
#transformers #pytorch #safetensors #bert #token-classification #et #base_model-tartuNLP/EstBERT #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 1e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 1024
* optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
* lr\_scheduler\_type: polynomial
* max num\_epochs: 150
* early stopping limit: 20
* early stopping tol: 0.0001
* mixed\_precision\_training: Native AMP ### Training results


The final model was saved after epoch 53 (shown in bold) where the overall F1 was the highest on the development set. ### Framework versions


* Transformers 4.16.2
* Pytorch 1.10.2+cu113
* Datasets 1.18.3
* Tokenizers 0.11.0
passage: TAGS
#transformers #pytorch #safetensors #bert #token-classification #et #base_model-tartuNLP/EstBERT #license-cc-by-4.0 #autotrain_compatible #endpoints_compatible #region-us 
### Training hyperparameters


The following hyperparameters were used during training:


* learning\_rate: 1e-05
* train\_batch\_size: 16
* eval\_batch\_size: 16
* seed: 1024
* optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
* lr\_scheduler\_type: polynomial
* max num\_epochs: 150
* early stopping limit: 20
* early stopping tol: 0.0001
* mixed\_precision\_training: Native AMP### Training results


The final model was saved after epoch 53 (shown in bold) where the overall F1 was the highest on the development set.### Framework versions


* Transformers 4.16.2
* Pytorch 1.10.2+cu113
* Datasets 1.18.3
* Tokenizers 0.11.0