layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7045
• Answer: {'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809}
• Header: {'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119}
• Question: {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}
• Overall Precision: 0.7148
• Overall Recall: 0.7772
• Overall F1: 0.7447
• Overall Accuracy: 0.8045

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8311	1.0	10	1.5893	{'precision': 0.01643192488262911, 'recall': 0.0173053152039555, 'f1': 0.016857314870559904, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.23476005188067445, 'recall': 0.1699530516431925, 'f1': 0.19716775599128541, 'number': 1065}	0.1201	0.0978	0.1079	0.3735
1.453	2.0	20	1.2320	{'precision': 0.16265750286368844, 'recall': 0.17552533992583436, 'f1': 0.16884661117717004, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.43330980945659847, 'recall': 0.5765258215962441, 'f1': 0.49476228847703463, 'number': 1065}	0.3301	0.3793	0.3530	0.5937
1.0885	3.0	30	0.9242	{'precision': 0.4983991462113127, 'recall': 0.5772558714462299, 'f1': 0.5349369988545246, 'number': 809}	{'precision': 0.05263157894736842, 'recall': 0.008403361344537815, 'f1': 0.014492753623188406, 'number': 119}	{'precision': 0.556745182012848, 'recall': 0.7323943661971831, 'f1': 0.632603406326034, 'number': 1065}	0.5295	0.6262	0.5738	0.7132
0.8351	4.0	40	0.7984	{'precision': 0.5991902834008097, 'recall': 0.7317676143386898, 'f1': 0.6588759042849194, 'number': 809}	{'precision': 0.16, 'recall': 0.06722689075630252, 'f1': 0.09467455621301775, 'number': 119}	{'precision': 0.6562763268744735, 'recall': 0.7314553990610329, 'f1': 0.691829484902309, 'number': 1065}	0.6198	0.6919	0.6539	0.7574
0.6746	5.0	50	0.7364	{'precision': 0.6524663677130045, 'recall': 0.7194066749072929, 'f1': 0.6843033509700176, 'number': 809}	{'precision': 0.21951219512195122, 'recall': 0.15126050420168066, 'f1': 0.1791044776119403, 'number': 119}	{'precision': 0.6493212669683258, 'recall': 0.8084507042253521, 'f1': 0.7202007528230866, 'number': 1065}	0.6352	0.7331	0.6806	0.7789
0.5833	6.0	60	0.7065	{'precision': 0.6387487386478304, 'recall': 0.7824474660074165, 'f1': 0.7033333333333333, 'number': 809}	{'precision': 0.25333333333333335, 'recall': 0.15966386554621848, 'f1': 0.1958762886597938, 'number': 119}	{'precision': 0.7177489177489178, 'recall': 0.7784037558685446, 'f1': 0.7468468468468469, 'number': 1065}	0.6668	0.7431	0.7029	0.7837
0.5101	7.0	70	0.6765	{'precision': 0.6811751904243744, 'recall': 0.7737948084054388, 'f1': 0.724537037037037, 'number': 809}	{'precision': 0.2564102564102564, 'recall': 0.25210084033613445, 'f1': 0.2542372881355932, 'number': 119}	{'precision': 0.7319762510602206, 'recall': 0.8103286384976526, 'f1': 0.7691622103386809, 'number': 1065}	0.6858	0.7622	0.7220	0.7972
0.4538	8.0	80	0.6643	{'precision': 0.6775210084033614, 'recall': 0.7972805933250927, 'f1': 0.7325383304940376, 'number': 809}	{'precision': 0.23893805309734514, 'recall': 0.226890756302521, 'f1': 0.2327586206896552, 'number': 119}	{'precision': 0.7389830508474576, 'recall': 0.8187793427230047, 'f1': 0.7768374164810691, 'number': 1065}	0.6878	0.7747	0.7286	0.8014
0.3958	9.0	90	0.6724	{'precision': 0.7022222222222222, 'recall': 0.7812113720642769, 'f1': 0.7396138092451726, 'number': 809}	{'precision': 0.25757575757575757, 'recall': 0.2857142857142857, 'f1': 0.27091633466135456, 'number': 119}	{'precision': 0.7319932998324958, 'recall': 0.8206572769953052, 'f1': 0.7737937140327579, 'number': 1065}	0.6918	0.7727	0.7300	0.7994
0.3902	10.0	100	0.6726	{'precision': 0.6846071044133477, 'recall': 0.7861557478368356, 'f1': 0.7318757192174913, 'number': 809}	{'precision': 0.288135593220339, 'recall': 0.2857142857142857, 'f1': 0.2869198312236287, 'number': 119}	{'precision': 0.7652790079716564, 'recall': 0.8112676056338028, 'f1': 0.7876025524156791, 'number': 1065}	0.7050	0.7697	0.7359	0.8080
0.3294	11.0	110	0.6827	{'precision': 0.7018701870187019, 'recall': 0.788627935723115, 'f1': 0.7427240977881256, 'number': 809}	{'precision': 0.28888888888888886, 'recall': 0.3277310924369748, 'f1': 0.3070866141732283, 'number': 119}	{'precision': 0.7508561643835616, 'recall': 0.8234741784037559, 'f1': 0.7854903716972683, 'number': 1065}	0.7025	0.7797	0.7391	0.8032
0.3124	12.0	120	0.6909	{'precision': 0.6974697469746974, 'recall': 0.7836835599505563, 'f1': 0.7380675203725262, 'number': 809}	{'precision': 0.3125, 'recall': 0.33613445378151263, 'f1': 0.3238866396761134, 'number': 119}	{'precision': 0.771960958296362, 'recall': 0.8169014084507042, 'f1': 0.7937956204379562, 'number': 1065}	0.7135	0.7747	0.7428	0.8047
0.2965	13.0	130	0.6986	{'precision': 0.7002212389380531, 'recall': 0.7824474660074165, 'f1': 0.7390542907180385, 'number': 809}	{'precision': 0.3230769230769231, 'recall': 0.35294117647058826, 'f1': 0.3373493975903615, 'number': 119}	{'precision': 0.7712014134275619, 'recall': 0.819718309859155, 'f1': 0.7947200728265817, 'number': 1065}	0.7147	0.7767	0.7444	0.8040
0.2676	14.0	140	0.7010	{'precision': 0.7028824833702882, 'recall': 0.7836835599505563, 'f1': 0.7410870835768557, 'number': 809}	{'precision': 0.32575757575757575, 'recall': 0.36134453781512604, 'f1': 0.3426294820717131, 'number': 119}	{'precision': 0.7768888888888889, 'recall': 0.8206572769953052, 'f1': 0.7981735159817351, 'number': 1065}	0.7184	0.7782	0.7471	0.8060
0.2747	15.0	150	0.7045	{'precision': 0.7013274336283186, 'recall': 0.7836835599505563, 'f1': 0.7402218330414477, 'number': 809}	{'precision': 0.3111111111111111, 'recall': 0.35294117647058826, 'f1': 0.33070866141732286, 'number': 119}	{'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}	0.7148	0.7772	0.7447	0.8045

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1