layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7211
  • Answer: {'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809}
  • Header: {'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119}
  • Question: {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065}
  • Overall Precision: 0.7332
  • Overall Recall: 0.8013
  • Overall F1: 0.7658
  • Overall Accuracy: 0.7963

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 15
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Answer Header Question Overall Precision Overall Recall Overall F1 Overall Accuracy
1.7901 1.0 10 1.5938 {'precision': 0.017361111111111112, 'recall': 0.012360939431396786, 'f1': 0.014440433212996389, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.32525252525252524, 'recall': 0.1511737089201878, 'f1': 0.20641025641025643, 'number': 1065} 0.1597 0.0858 0.1116 0.3415
1.4447 2.0 20 1.2469 {'precision': 0.22497522299306244, 'recall': 0.28059332509270707, 'f1': 0.24972497249724973, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.4132890365448505, 'recall': 0.584037558685446, 'f1': 0.4840466926070039, 'number': 1065} 0.3377 0.4260 0.3767 0.5933
1.0816 3.0 30 0.9331 {'precision': 0.5004995004995005, 'recall': 0.619283065512979, 'f1': 0.5535911602209945, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.5584415584415584, 'recall': 0.7267605633802817, 'f1': 0.631578947368421, 'number': 1065} 0.5260 0.6397 0.5773 0.7095
0.8262 4.0 40 0.7964 {'precision': 0.5772357723577236, 'recall': 0.7021013597033374, 'f1': 0.633575013943112, 'number': 809} {'precision': 0.15625, 'recall': 0.08403361344537816, 'f1': 0.10928961748633881, 'number': 119} {'precision': 0.6492659053833605, 'recall': 0.7474178403755869, 'f1': 0.6948930597992143, 'number': 1065} 0.6042 0.6894 0.6440 0.7450
0.6674 5.0 50 0.7441 {'precision': 0.6445182724252492, 'recall': 0.7194066749072929, 'f1': 0.6799065420560747, 'number': 809} {'precision': 0.22105263157894736, 'recall': 0.17647058823529413, 'f1': 0.19626168224299065, 'number': 119} {'precision': 0.6424870466321243, 'recall': 0.8150234741784037, 'f1': 0.7185430463576157, 'number': 1065} 0.6262 0.7381 0.6776 0.7673
0.5736 6.0 60 0.7005 {'precision': 0.6451942740286298, 'recall': 0.7799752781211372, 'f1': 0.7062115277000558, 'number': 809} {'precision': 0.20454545454545456, 'recall': 0.15126050420168066, 'f1': 0.17391304347826086, 'number': 119} {'precision': 0.7412891986062717, 'recall': 0.7990610328638498, 'f1': 0.7690917306823317, 'number': 1065} 0.6775 0.7526 0.7131 0.7755
0.5042 7.0 70 0.6801 {'precision': 0.6768743400211193, 'recall': 0.792336217552534, 'f1': 0.7300683371298405, 'number': 809} {'precision': 0.22018348623853212, 'recall': 0.20168067226890757, 'f1': 0.21052631578947367, 'number': 119} {'precision': 0.7412765957446809, 'recall': 0.8178403755868544, 'f1': 0.7776785714285714, 'number': 1065} 0.6885 0.7707 0.7273 0.7841
0.4479 8.0 80 0.6712 {'precision': 0.6687565308254964, 'recall': 0.7911001236093943, 'f1': 0.7248018120045301, 'number': 809} {'precision': 0.20610687022900764, 'recall': 0.226890756302521, 'f1': 0.21600000000000003, 'number': 119} {'precision': 0.7404006677796328, 'recall': 0.8328638497652582, 'f1': 0.7839151568714097, 'number': 1065} 0.6798 0.7797 0.7263 0.7900
0.3931 9.0 90 0.6806 {'precision': 0.7054263565891473, 'recall': 0.7873918417799752, 'f1': 0.7441588785046728, 'number': 809} {'precision': 0.2809917355371901, 'recall': 0.2857142857142857, 'f1': 0.2833333333333333, 'number': 119} {'precision': 0.7510620220900595, 'recall': 0.8300469483568075, 'f1': 0.7885816235504014, 'number': 1065} 0.7065 0.7802 0.7415 0.7955
0.3875 10.0 100 0.6819 {'precision': 0.7014767932489452, 'recall': 0.8220024721878862, 'f1': 0.7569721115537849, 'number': 809} {'precision': 0.3, 'recall': 0.3025210084033613, 'f1': 0.301255230125523, 'number': 119} {'precision': 0.7633851468048359, 'recall': 0.8300469483568075, 'f1': 0.7953216374269007, 'number': 1065} 0.7120 0.7953 0.7514 0.7952
0.3309 11.0 110 0.7016 {'precision': 0.7204419889502762, 'recall': 0.8059332509270705, 'f1': 0.7607934655775962, 'number': 809} {'precision': 0.2949640287769784, 'recall': 0.3445378151260504, 'f1': 0.31782945736434104, 'number': 119} {'precision': 0.7535864978902953, 'recall': 0.8384976525821596, 'f1': 0.7937777777777778, 'number': 1065} 0.7115 0.7958 0.7513 0.7969
0.3142 12.0 120 0.7081 {'precision': 0.7178924259055982, 'recall': 0.8084054388133498, 'f1': 0.7604651162790698, 'number': 809} {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119} {'precision': 0.7768014059753954, 'recall': 0.8300469483568075, 'f1': 0.8025419881979118, 'number': 1065} 0.7245 0.7918 0.7567 0.7993
0.2992 13.0 130 0.7160 {'precision': 0.716304347826087, 'recall': 0.8145859085290482, 'f1': 0.7622903412377097, 'number': 809} {'precision': 0.304, 'recall': 0.31932773109243695, 'f1': 0.31147540983606553, 'number': 119} {'precision': 0.7796167247386759, 'recall': 0.8403755868544601, 'f1': 0.8088567555354722, 'number': 1065} 0.7259 0.7988 0.7606 0.7938
0.2746 14.0 140 0.7194 {'precision': 0.7238723872387238, 'recall': 0.8133498145859085, 'f1': 0.7660069848661233, 'number': 809} {'precision': 0.32061068702290074, 'recall': 0.35294117647058826, 'f1': 0.336, 'number': 119} {'precision': 0.7859030837004405, 'recall': 0.8375586854460094, 'f1': 0.8109090909090909, 'number': 1065} 0.7320 0.7988 0.7639 0.7957
0.2735 15.0 150 0.7211 {'precision': 0.7268722466960352, 'recall': 0.8158220024721878, 'f1': 0.7687827606290041, 'number': 809} {'precision': 0.31343283582089554, 'recall': 0.35294117647058826, 'f1': 0.3320158102766798, 'number': 119} {'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065} 0.7332 0.8013 0.7658 0.7963

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
9
Safetensors
Model size
113M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for sobey25701/layoutlm-funsd

Finetuned
(162)
this model