layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7045
  • Answer: {'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809}
  • Header: {'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119}
  • Question: {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}
  • Overall Precision: 0.7148
  • Overall Recall: 0.7772
  • Overall F1: 0.7447
  • Overall Accuracy: 0.8045

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 15
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Answer Header Question Overall Precision Overall Recall Overall F1 Overall Accuracy
1.8311 1.0 10 1.5893 {'precision': 0.01643192488262911, 'recall': 0.0173053152039555, 'f1': 0.016857314870559904, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.23476005188067445, 'recall': 0.1699530516431925, 'f1': 0.19716775599128541, 'number': 1065} 0.1201 0.0978 0.1079 0.3735
1.453 2.0 20 1.2320 {'precision': 0.16265750286368844, 'recall': 0.17552533992583436, 'f1': 0.16884661117717004, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.43330980945659847, 'recall': 0.5765258215962441, 'f1': 0.49476228847703463, 'number': 1065} 0.3301 0.3793 0.3530 0.5937
1.0885 3.0 30 0.9242 {'precision': 0.4983991462113127, 'recall': 0.5772558714462299, 'f1': 0.5349369988545246, 'number': 809} {'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119} {'precision': 0.556745182012848, 'recall': 0.7323943661971831, 'f1': 0.632603406326034, 'number': 1065} 0.5295 0.6262 0.5738 0.7132
0.8351 4.0 40 0.7984 {'precision': 0.5991902834008097, 'recall': 0.7317676143386898, 'f1': 0.6588759042849194, 'number': 809} {'precision': 0.16, 'recall': 0.06722689075630252, 'f1': 0.09467455621301775, 'number': 119} {'precision': 0.6562763268744735, 'recall': 0.7314553990610329, 'f1': 0.691829484902309, 'number': 1065} 0.6198 0.6919 0.6539 0.7574
0.6746 5.0 50 0.7364 {'precision': 0.6524663677130045, 'recall': 0.7194066749072929, 'f1': 0.6843033509700176, 'number': 809} {'precision': 0.21951219512195122, 'recall': 0.15126050420168066, 'f1': 0.1791044776119403, 'number': 119} {'precision': 0.6493212669683258, 'recall': 0.8084507042253521, 'f1': 0.7202007528230866, 'number': 1065} 0.6352 0.7331 0.6806 0.7789
0.5833 6.0 60 0.7065 {'precision': 0.6387487386478304, 'recall': 0.7824474660074165, 'f1': 0.7033333333333333, 'number': 809} {'precision': 0.25333333333333335, 'recall': 0.15966386554621848, 'f1': 0.1958762886597938, 'number': 119} {'precision': 0.7177489177489178, 'recall': 0.7784037558685446, 'f1': 0.7468468468468469, 'number': 1065} 0.6668 0.7431 0.7029 0.7837
0.5101 7.0 70 0.6765 {'precision': 0.6811751904243744, 'recall': 0.7737948084054388, 'f1': 0.724537037037037, 'number': 809} {'precision': 0.2564102564102564, 'recall': 0.25210084033613445, 'f1': 0.2542372881355932, 'number': 119} {'precision': 0.7319762510602206, 'recall': 0.8103286384976526, 'f1': 0.7691622103386809, 'number': 1065} 0.6858 0.7622 0.7220 0.7972
0.4538 8.0 80 0.6643 {'precision': 0.6775210084033614, 'recall': 0.7972805933250927, 'f1': 0.7325383304940376, 'number': 809} {'precision': 0.23893805309734514, 'recall': 0.226890756302521, 'f1': 0.2327586206896552, 'number': 119} {'precision': 0.7389830508474576, 'recall': 0.8187793427230047, 'f1': 0.7768374164810691, 'number': 1065} 0.6878 0.7747 0.7286 0.8014
0.3958 9.0 90 0.6724 {'precision': 0.7022222222222222, 'recall': 0.7812113720642769, 'f1': 0.7396138092451726, 'number': 809} {'precision': 0.25757575757575757, 'recall': 0.2857142857142857, 'f1': 0.27091633466135456, 'number': 119} {'precision': 0.7319932998324958, 'recall': 0.8206572769953052, 'f1': 0.7737937140327579, 'number': 1065} 0.6918 0.7727 0.7300 0.7994
0.3902 10.0 100 0.6726 {'precision': 0.6846071044133477, 'recall': 0.7861557478368356, 'f1': 0.7318757192174913, 'number': 809} {'precision': 0.288135593220339, 'recall': 0.2857142857142857, 'f1': 0.2869198312236287, 'number': 119} {'precision': 0.7652790079716564, 'recall': 0.8112676056338028, 'f1': 0.7876025524156791, 'number': 1065} 0.7050 0.7697 0.7359 0.8080
0.3294 11.0 110 0.6827 {'precision': 0.7018701870187019, 'recall': 0.788627935723115, 'f1': 0.7427240977881256, 'number': 809} {'precision': 0.28888888888888886, 'recall': 0.3277310924369748, 'f1': 0.3070866141732283, 'number': 119} {'precision': 0.7508561643835616, 'recall': 0.8234741784037559, 'f1': 0.7854903716972683, 'number': 1065} 0.7025 0.7797 0.7391 0.8032
0.3124 12.0 120 0.6909 {'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809} {'precision': 0.3125, 'recall': 0.33613445378151263, 'f1': 0.3238866396761134, 'number': 119} {'precision': 0.771960958296362, 'recall': 0.8169014084507042, 'f1': 0.7937956204379562, 'number': 1065} 0.7135 0.7747 0.7428 0.8047
0.2965 13.0 130 0.6986 {'precision': 0.7002212389380531, 'recall': 0.7824474660074165, 'f1': 0.7390542907180385, 'number': 809} {'precision': 0.3230769230769231, 'recall': 0.35294117647058826, 'f1': 0.3373493975903615, 'number': 119} {'precision': 0.7712014134275619, 'recall': 0.819718309859155, 'f1': 0.7947200728265817, 'number': 1065} 0.7147 0.7767 0.7444 0.8040
0.2676 14.0 140 0.7010 {'precision': 0.7028824833702882, 'recall': 0.7836835599505563, 'f1': 0.7410870835768557, 'number': 809} {'precision': 0.32575757575757575, 'recall': 0.36134453781512604, 'f1': 0.3426294820717131, 'number': 119} {'precision': 0.7768888888888889, 'recall': 0.8206572769953052, 'f1': 0.7981735159817351, 'number': 1065} 0.7184 0.7782 0.7471 0.8060
0.2747 15.0 150 0.7045 {'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809} {'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119} {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065} 0.7148 0.7772 0.7447 0.8045

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
4
Safetensors
Model size
113M params
Tensor type
F32
·
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.

Model tree for yadavom/layoutlm-funsd

Finetuned
(162)
this model