File size: 20,758 Bytes
95a4d4c b9311e0 95a4d4c b9311e0 95a4d4c b9311e0 95a4d4c b9311e0 95a4d4c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd3
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd3
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8917
- Answer: {'precision': 0.7259507829977628, 'recall': 0.8022249690976514, 'f1': 0.7621843805049912, 'number': 809}
- Header: {'precision': 0.3355263157894737, 'recall': 0.42857142857142855, 'f1': 0.3763837638376384, 'number': 119}
- Question: {'precision': 0.7875, 'recall': 0.828169014084507, 'f1': 0.8073226544622427, 'number': 1065}
- Overall Precision: 0.7304
- Overall Recall: 0.7938
- Overall F1: 0.7608
- Overall Accuracy: 0.7919
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 40
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.9148 | 1.0 | 10 | 1.9348 | {'precision': 0.02721922511034821, 'recall': 0.13720642768850433, 'f1': 0.04542664211172499, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.040596094552929084, 'recall': 0.07417840375586854, 'f1': 0.0524742610428429, 'number': 1065} | 0.0308 | 0.0953 | 0.0466 | 0.1513 |
| 1.9105 | 2.0 | 20 | 1.9240 | {'precision': 0.02663934426229508, 'recall': 0.12855377008652658, 'f1': 0.04413324846170167, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.04175152749490835, 'recall': 0.07699530516431925, 'f1': 0.05414328161109277, 'number': 1065} | 0.0310 | 0.0933 | 0.0466 | 0.1596 |
| 1.8962 | 3.0 | 30 | 1.9059 | {'precision': 0.025634033269702754, 'recall': 0.1161928306551298, 'f1': 0.0420017873100983, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.04310344827586207, 'recall': 0.07981220657276995, 'f1': 0.05597629239380968, 'number': 1065} | 0.0312 | 0.0898 | 0.0464 | 0.1729 |
| 1.8763 | 4.0 | 40 | 1.8818 | {'precision': 0.025390027531355153, 'recall': 0.10259579728059333, 'f1': 0.04070622854340363, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.04250907205806117, 'recall': 0.07699530516431925, 'f1': 0.05477621910487642, 'number': 1065} | 0.0314 | 0.0828 | 0.0456 | 0.1906 |
| 1.8461 | 5.0 | 50 | 1.8523 | {'precision': 0.026613197229310975, 'recall': 0.09023485784919653, 'f1': 0.0411036036036036, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.04662643993417444, 'recall': 0.07981220657276995, 'f1': 0.05886426592797784, 'number': 1065} | 0.0344 | 0.0793 | 0.0480 | 0.2194 |
| 1.816 | 6.0 | 60 | 1.8190 | {'precision': 0.027329749103942653, 'recall': 0.0754017305315204, 'f1': 0.04011838211114765, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.05275779376498801, 'recall': 0.08262910798122065, 'f1': 0.06439809732894256, 'number': 1065} | 0.0381 | 0.0748 | 0.0505 | 0.2467 |
| 1.7864 | 7.0 | 70 | 1.7834 | {'precision': 0.022129186602870814, 'recall': 0.04573547589616811, 'f1': 0.029826682789197905, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.06278026905829596, 'recall': 0.07887323943661972, 'f1': 0.06991260923845194, 'number': 1065} | 0.0402 | 0.0607 | 0.0484 | 0.2767 |
| 1.7409 | 8.0 | 80 | 1.7413 | {'precision': 0.024390243902439025, 'recall': 0.038318912237330034, 'f1': 0.029807692307692306, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.07393715341959335, 'recall': 0.07511737089201878, 'f1': 0.07452258965999067, 'number': 1065} | 0.0471 | 0.0557 | 0.0511 | 0.2975 |
| 1.6984 | 9.0 | 90 | 1.6959 | {'precision': 0.01787487586891758, 'recall': 0.022249690976514216, 'f1': 0.019823788546255508, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.10549943883277217, 'recall': 0.08826291079812207, 'f1': 0.09611451942740287, 'number': 1065} | 0.0590 | 0.0562 | 0.0576 | 0.3175 |
| 1.6582 | 10.0 | 100 | 1.6434 | {'precision': 0.02877697841726619, 'recall': 0.034610630407911, 'f1': 0.031425364758698095, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1774033696729435, 'recall': 0.168075117370892, 'f1': 0.1726133076181292, 'number': 1065} | 0.1044 | 0.1039 | 0.1042 | 0.3466 |
| 1.5875 | 11.0 | 110 | 1.5763 | {'precision': 0.04632152588555858, 'recall': 0.0630407911001236, 'f1': 0.05340314136125654, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.23351023502653526, 'recall': 0.2892018779342723, 'f1': 0.25838926174496646, 'number': 1065} | 0.1483 | 0.1801 | 0.1627 | 0.4021 |
| 1.513 | 12.0 | 120 | 1.4936 | {'precision': 0.06457739791073125, 'recall': 0.08405438813349815, 'f1': 0.07303974221267456, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.29396325459317585, 'recall': 0.42065727699530514, 'f1': 0.3460795674005407, 'number': 1065} | 0.2002 | 0.2589 | 0.2258 | 0.4569 |
| 1.425 | 13.0 | 130 | 1.3967 | {'precision': 0.08278867102396514, 'recall': 0.09394313967861558, 'f1': 0.08801389693109439, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.36175882744836774, 'recall': 0.5098591549295775, 'f1': 0.4232268121590023, 'number': 1065} | 0.2559 | 0.3106 | 0.2806 | 0.4954 |
| 1.2919 | 14.0 | 140 | 1.2813 | {'precision': 0.14430379746835442, 'recall': 0.14091470951792337, 'f1': 0.1425891181988743, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4328882642304989, 'recall': 0.5784037558685446, 'f1': 0.4951768488745981, 'number': 1065} | 0.3299 | 0.3663 | 0.3471 | 0.5484 |
| 1.187 | 15.0 | 150 | 1.1608 | {'precision': 0.24360699865410498, 'recall': 0.22373300370828184, 'f1': 0.23324742268041238, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5067567567567568, 'recall': 0.6338028169014085, 'f1': 0.5632040050062578, 'number': 1065} | 0.4125 | 0.4295 | 0.4208 | 0.5971 |
| 1.0614 | 16.0 | 160 | 1.0322 | {'precision': 0.3975609756097561, 'recall': 0.40296662546353523, 'f1': 0.40024554941682017, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5522949586155004, 'recall': 0.6892018779342723, 'f1': 0.6131996658312447, 'number': 1065} | 0.4921 | 0.5319 | 0.5112 | 0.6638 |
| 0.9293 | 17.0 | 170 | 0.9263 | {'precision': 0.5032537960954447, 'recall': 0.5735475896168108, 'f1': 0.5361062969381861, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5963302752293578, 'recall': 0.7323943661971831, 'f1': 0.6573957016434893, 'number': 1065} | 0.5531 | 0.6242 | 0.5865 | 0.7108 |
| 0.8159 | 18.0 | 180 | 0.8344 | {'precision': 0.5733610822060354, 'recall': 0.681087762669963, 'f1': 0.6225988700564972, 'number': 809} | {'precision': 0.08, 'recall': 0.03361344537815126, 'f1': 0.047337278106508875, 'number': 119} | {'precision': 0.6435483870967742, 'recall': 0.7492957746478873, 'f1': 0.6924078091106292, 'number': 1065} | 0.6011 | 0.6789 | 0.6376 | 0.7407 |
| 0.7244 | 19.0 | 190 | 0.7626 | {'precision': 0.6211312700106724, 'recall': 0.7194066749072929, 'f1': 0.6666666666666666, 'number': 809} | {'precision': 0.18867924528301888, 'recall': 0.08403361344537816, 'f1': 0.11627906976744187, 'number': 119} | {'precision': 0.6724422442244224, 'recall': 0.7652582159624414, 'f1': 0.7158541941150638, 'number': 1065} | 0.6390 | 0.7060 | 0.6708 | 0.7603 |
| 0.6506 | 20.0 | 200 | 0.7306 | {'precision': 0.6355748373101953, 'recall': 0.7243510506798516, 'f1': 0.6770652801848642, 'number': 809} | {'precision': 0.2222222222222222, 'recall': 0.15126050420168066, 'f1': 0.18, 'number': 119} | {'precision': 0.66953125, 'recall': 0.8046948356807512, 'f1': 0.7309168443496802, 'number': 1065} | 0.6399 | 0.7331 | 0.6833 | 0.7771 |
| 0.588 | 21.0 | 210 | 0.6980 | {'precision': 0.6449197860962567, 'recall': 0.7453646477132262, 'f1': 0.6915137614678899, 'number': 809} | {'precision': 0.22340425531914893, 'recall': 0.17647058823529413, 'f1': 0.1971830985915493, 'number': 119} | {'precision': 0.6994171523730225, 'recall': 0.7887323943661971, 'f1': 0.7413945278022949, 'number': 1065} | 0.6565 | 0.7346 | 0.6933 | 0.7812 |
| 0.552 | 22.0 | 220 | 0.6761 | {'precision': 0.6673819742489271, 'recall': 0.7688504326328801, 'f1': 0.7145318782309018, 'number': 809} | {'precision': 0.22330097087378642, 'recall': 0.19327731092436976, 'f1': 0.20720720720720723, 'number': 119} | {'precision': 0.713469387755102, 'recall': 0.8206572769953052, 'f1': 0.7633187772925764, 'number': 1065} | 0.6721 | 0.7622 | 0.7143 | 0.7907 |
| 0.4842 | 23.0 | 230 | 0.6748 | {'precision': 0.6846846846846847, 'recall': 0.7515451174289246, 'f1': 0.7165586328815557, 'number': 809} | {'precision': 0.25961538461538464, 'recall': 0.226890756302521, 'f1': 0.242152466367713, 'number': 119} | {'precision': 0.7347789824854045, 'recall': 0.8272300469483568, 'f1': 0.7782685512367491, 'number': 1065} | 0.6919 | 0.7607 | 0.7247 | 0.7955 |
| 0.4503 | 24.0 | 240 | 0.6768 | {'precision': 0.6757322175732218, 'recall': 0.7985166872682324, 'f1': 0.7320113314447593, 'number': 809} | {'precision': 0.26732673267326734, 'recall': 0.226890756302521, 'f1': 0.24545454545454548, 'number': 119} | {'precision': 0.7468460891505467, 'recall': 0.8338028169014085, 'f1': 0.7879325643300797, 'number': 1065} | 0.6950 | 0.7832 | 0.7365 | 0.7949 |
| 0.4104 | 25.0 | 250 | 0.6905 | {'precision': 0.6844444444444444, 'recall': 0.761433868974042, 'f1': 0.7208894090111176, 'number': 809} | {'precision': 0.2689075630252101, 'recall': 0.2689075630252101, 'f1': 0.2689075630252101, 'number': 119} | {'precision': 0.7474747474747475, 'recall': 0.8338028169014085, 'f1': 0.788282290279627, 'number': 1065} | 0.6960 | 0.7707 | 0.7314 | 0.7921 |
| 0.3846 | 26.0 | 260 | 0.6857 | {'precision': 0.6980920314253648, 'recall': 0.7688504326328801, 'f1': 0.731764705882353, 'number': 809} | {'precision': 0.2773109243697479, 'recall': 0.2773109243697479, 'f1': 0.2773109243697479, 'number': 119} | {'precision': 0.7468566638725901, 'recall': 0.8366197183098592, 'f1': 0.7891939769707705, 'number': 1065} | 0.7018 | 0.7757 | 0.7369 | 0.7966 |
| 0.3533 | 27.0 | 270 | 0.6714 | {'precision': 0.6938997821350763, 'recall': 0.7873918417799752, 'f1': 0.7376954255935148, 'number': 809} | {'precision': 0.2767857142857143, 'recall': 0.2605042016806723, 'f1': 0.2683982683982684, 'number': 119} | {'precision': 0.7580645161290323, 'recall': 0.8384976525821596, 'f1': 0.7962550156041017, 'number': 1065} | 0.7070 | 0.7832 | 0.7432 | 0.7971 |
| 0.3271 | 28.0 | 280 | 0.7090 | {'precision': 0.6864035087719298, 'recall': 0.7737948084054388, 'f1': 0.7274840209180709, 'number': 809} | {'precision': 0.2706766917293233, 'recall': 0.3025210084033613, 'f1': 0.28571428571428564, 'number': 119} | {'precision': 0.7462562396006656, 'recall': 0.8422535211267606, 'f1': 0.7913542126157919, 'number': 1065} | 0.6938 | 0.7822 | 0.7354 | 0.7929 |
| 0.3031 | 29.0 | 290 | 0.7212 | {'precision': 0.7275943396226415, 'recall': 0.7626699629171817, 'f1': 0.7447193723596861, 'number': 809} | {'precision': 0.3130434782608696, 'recall': 0.3025210084033613, 'f1': 0.3076923076923077, 'number': 119} | {'precision': 0.7788546255506608, 'recall': 0.8300469483568075, 'f1': 0.8036363636363637, 'number': 1065} | 0.7326 | 0.7712 | 0.7514 | 0.7949 |
| 0.2723 | 30.0 | 300 | 0.7351 | {'precision': 0.7036637931034483, 'recall': 0.8071693448702101, 'f1': 0.7518710420264824, 'number': 809} | {'precision': 0.2740740740740741, 'recall': 0.31092436974789917, 'f1': 0.29133858267716534, 'number': 119} | {'precision': 0.7715289982425307, 'recall': 0.8244131455399061, 'f1': 0.7970948706309579, 'number': 1065} | 0.7124 | 0.7868 | 0.7477 | 0.7982 |
| 0.2589 | 31.0 | 310 | 0.7356 | {'precision': 0.6878914405010439, 'recall': 0.8145859085290482, 'f1': 0.745897000565931, 'number': 809} | {'precision': 0.30973451327433627, 'recall': 0.29411764705882354, 'f1': 0.3017241379310345, 'number': 119} | {'precision': 0.7708688245315162, 'recall': 0.8497652582159625, 'f1': 0.8083966056275124, 'number': 1065} | 0.7122 | 0.8023 | 0.7546 | 0.7948 |
| 0.2305 | 32.0 | 320 | 0.7378 | {'precision': 0.7220956719817767, 'recall': 0.7836835599505563, 'f1': 0.7516301126259632, 'number': 809} | {'precision': 0.32432432432432434, 'recall': 0.3025210084033613, 'f1': 0.31304347826086953, 'number': 119} | {'precision': 0.7734711455641688, 'recall': 0.8431924882629108, 'f1': 0.8068283917340521, 'number': 1065} | 0.7293 | 0.7868 | 0.7569 | 0.7983 |
| 0.2114 | 33.0 | 330 | 0.7546 | {'precision': 0.7093275488069414, 'recall': 0.8084054388133498, 'f1': 0.7556325823223571, 'number': 809} | {'precision': 0.3392857142857143, 'recall': 0.31932773109243695, 'f1': 0.32900432900432897, 'number': 119} | {'precision': 0.7737162750217581, 'recall': 0.8347417840375587, 'f1': 0.803071364046974, 'number': 1065} | 0.7242 | 0.7933 | 0.7572 | 0.7987 |
| 0.1921 | 34.0 | 340 | 0.7701 | {'precision': 0.724669603524229, 'recall': 0.8133498145859085, 'f1': 0.7664531158998252, 'number': 809} | {'precision': 0.2980132450331126, 'recall': 0.37815126050420167, 'f1': 0.33333333333333337, 'number': 119} | {'precision': 0.7862939585211902, 'recall': 0.8187793427230047, 'f1': 0.8022079116835327, 'number': 1065} | 0.7265 | 0.7903 | 0.7570 | 0.7991 |
| 0.1791 | 35.0 | 350 | 0.8101 | {'precision': 0.7331812998859749, 'recall': 0.7948084054388134, 'f1': 0.7627520759193357, 'number': 809} | {'precision': 0.31724137931034485, 'recall': 0.3865546218487395, 'f1': 0.3484848484848485, 'number': 119} | {'precision': 0.7892416225749559, 'recall': 0.8403755868544601, 'f1': 0.814006366530241, 'number': 1065} | 0.7347 | 0.7948 | 0.7636 | 0.7955 |
| 0.1607 | 36.0 | 360 | 0.7987 | {'precision': 0.7369020501138952, 'recall': 0.799752781211372, 'f1': 0.7670420865441612, 'number': 809} | {'precision': 0.31690140845070425, 'recall': 0.37815126050420167, 'f1': 0.3448275862068965, 'number': 119} | {'precision': 0.784121320249777, 'recall': 0.8253521126760563, 'f1': 0.8042086001829827, 'number': 1065} | 0.7338 | 0.7883 | 0.7600 | 0.8035 |
| 0.145 | 37.0 | 370 | 0.8154 | {'precision': 0.7079741379310345, 'recall': 0.8121137206427689, 'f1': 0.7564766839378237, 'number': 809} | {'precision': 0.3248407643312102, 'recall': 0.42857142857142855, 'f1': 0.3695652173913043, 'number': 119} | {'precision': 0.7934782608695652, 'recall': 0.8225352112676056, 'f1': 0.8077455048409405, 'number': 1065} | 0.7236 | 0.7948 | 0.7575 | 0.8013 |
| 0.139 | 38.0 | 380 | 0.8250 | {'precision': 0.7334083239595051, 'recall': 0.8059332509270705, 'f1': 0.767962308598351, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.4117647058823529, 'f1': 0.36842105263157887, 'number': 119} | {'precision': 0.7953110910730388, 'recall': 0.828169014084507, 'f1': 0.8114075436982521, 'number': 1065} | 0.7380 | 0.7943 | 0.7651 | 0.8062 |
| 0.1266 | 39.0 | 390 | 0.8796 | {'precision': 0.762962962962963, 'recall': 0.7639060568603214, 'f1': 0.7634342186534898, 'number': 809} | {'precision': 0.34591194968553457, 'recall': 0.46218487394957986, 'f1': 0.39568345323741, 'number': 119} | {'precision': 0.7884267631103075, 'recall': 0.8187793427230047, 'f1': 0.8033164440350069, 'number': 1065} | 0.7446 | 0.7752 | 0.7596 | 0.7989 |
| 0.1147 | 40.0 | 400 | 0.8917 | {'precision': 0.7259507829977628, 'recall': 0.8022249690976514, 'f1': 0.7621843805049912, 'number': 809} | {'precision': 0.3355263157894737, 'recall': 0.42857142857142855, 'f1': 0.3763837638376384, 'number': 119} | {'precision': 0.7875, 'recall': 0.828169014084507, 'f1': 0.8073226544622427, 'number': 1065} | 0.7304 | 0.7938 | 0.7608 | 0.7919 |
### Framework versions
- Transformers 4.41.2
- Pytorch 2.3.1+cu121
- Datasets 2.19.2
- Tokenizers 0.19.1
|