layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7033
• Answer: {'precision': 0.7135016465422612, 'recall': 0.8034610630407911, 'f1': 0.7558139534883722, 'number': 809}
• Header: {'precision': 0.34959349593495936, 'recall': 0.36134453781512604, 'f1': 0.35537190082644626, 'number': 119}
• Question: {'precision': 0.7895204262877442, 'recall': 0.8347417840375587, 'f1': 0.8115015974440895, 'number': 1065}
• Overall Precision: 0.7324
• Overall Recall: 0.7938
• Overall F1: 0.7619
• Overall Accuracy: 0.8096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7852	1.0	10	1.6133	{'precision': 0.0075, 'recall': 0.003708281829419036, 'f1': 0.004962779156327543, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.27348066298342544, 'recall': 0.09295774647887324, 'f1': 0.13875262789067977, 'number': 1065}	0.1339	0.0512	0.0740	0.3219
1.4748	2.0	20	1.2897	{'precision': 0.13947696139476962, 'recall': 0.138442521631644, 'f1': 0.13895781637717122, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4375, 'recall': 0.5126760563380282, 'f1': 0.4721141374837873, 'number': 1065}	0.3208	0.3302	0.3254	0.5949
1.1297	3.0	30	0.9650	{'precision': 0.4776931447225245, 'recall': 0.5426452410383189, 'f1': 0.5081018518518519, 'number': 809}	{'precision': 0.14285714285714285, 'recall': 0.025210084033613446, 'f1': 0.04285714285714286, 'number': 119}	{'precision': 0.5609561752988048, 'recall': 0.6610328638497652, 'f1': 0.606896551724138, 'number': 1065}	0.5221	0.5750	0.5473	0.7046
0.8686	4.0	40	0.7908	{'precision': 0.6090534979423868, 'recall': 0.7317676143386898, 'f1': 0.664795058955643, 'number': 809}	{'precision': 0.17543859649122806, 'recall': 0.08403361344537816, 'f1': 0.11363636363636363, 'number': 119}	{'precision': 0.66431718061674, 'recall': 0.707981220657277, 'f1': 0.6854545454545454, 'number': 1065}	0.6266	0.6804	0.6524	0.7606
0.688	5.0	50	0.7202	{'precision': 0.6440677966101694, 'recall': 0.7515451174289246, 'f1': 0.6936679977181973, 'number': 809}	{'precision': 0.345679012345679, 'recall': 0.23529411764705882, 'f1': 0.27999999999999997, 'number': 119}	{'precision': 0.6864197530864198, 'recall': 0.7830985915492957, 'f1': 0.7315789473684212, 'number': 1065}	0.6562	0.7376	0.6945	0.7799
0.5806	6.0	60	0.6899	{'precision': 0.6547368421052632, 'recall': 0.7688504326328801, 'f1': 0.7072200113700967, 'number': 809}	{'precision': 0.3068181818181818, 'recall': 0.226890756302521, 'f1': 0.2608695652173913, 'number': 119}	{'precision': 0.723441615452151, 'recall': 0.7737089201877935, 'f1': 0.7477313974591652, 'number': 1065}	0.6766	0.7391	0.7065	0.7841
0.4975	7.0	70	0.6633	{'precision': 0.6722689075630253, 'recall': 0.7911001236093943, 'f1': 0.7268597387847815, 'number': 809}	{'precision': 0.30434782608695654, 'recall': 0.23529411764705882, 'f1': 0.2654028436018957, 'number': 119}	{'precision': 0.7429577464788732, 'recall': 0.7924882629107981, 'f1': 0.7669241253975465, 'number': 1065}	0.6936	0.7587	0.7247	0.7969
0.4425	8.0	80	0.6850	{'precision': 0.6790123456790124, 'recall': 0.8158220024721878, 'f1': 0.7411566535654126, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.29411764705882354, 'f1': 0.3125, 'number': 119}	{'precision': 0.7445319335083115, 'recall': 0.7990610328638498, 'f1': 0.7708333333333334, 'number': 1065}	0.6964	0.7757	0.7339	0.7933
0.3867	9.0	90	0.6700	{'precision': 0.7067254685777288, 'recall': 0.792336217552534, 'f1': 0.7470862470862472, 'number': 809}	{'precision': 0.3181818181818182, 'recall': 0.29411764705882354, 'f1': 0.3056768558951965, 'number': 119}	{'precision': 0.7567099567099567, 'recall': 0.8206572769953052, 'f1': 0.7873873873873873, 'number': 1065}	0.7136	0.7777	0.7443	0.8031
0.3746	10.0	100	0.6724	{'precision': 0.7029702970297029, 'recall': 0.7898640296662547, 'f1': 0.7438882421420255, 'number': 809}	{'precision': 0.3425925925925926, 'recall': 0.31092436974789917, 'f1': 0.3259911894273128, 'number': 119}	{'precision': 0.7665505226480837, 'recall': 0.8262910798122066, 'f1': 0.7953004970628107, 'number': 1065}	0.7187	0.7807	0.7484	0.8095
0.3198	11.0	110	0.6931	{'precision': 0.7155266015200868, 'recall': 0.8145859085290482, 'f1': 0.761849710982659, 'number': 809}	{'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119}	{'precision': 0.7760279965004374, 'recall': 0.8328638497652582, 'f1': 0.8034420289855072, 'number': 1065}	0.7237	0.7953	0.7578	0.8042
0.3041	12.0	120	0.6943	{'precision': 0.7182628062360802, 'recall': 0.7972805933250927, 'f1': 0.7557117750439368, 'number': 809}	{'precision': 0.33884297520661155, 'recall': 0.3445378151260504, 'f1': 0.3416666666666667, 'number': 119}	{'precision': 0.780053428317008, 'recall': 0.8225352112676056, 'f1': 0.8007312614259597, 'number': 1065}	0.7292	0.7837	0.7555	0.8068
0.2828	13.0	130	0.7021	{'precision': 0.7139737991266376, 'recall': 0.8084054388133498, 'f1': 0.7582608695652174, 'number': 809}	{'precision': 0.358974358974359, 'recall': 0.35294117647058826, 'f1': 0.35593220338983056, 'number': 119}	{'precision': 0.8, 'recall': 0.8413145539906103, 'f1': 0.820137299771167, 'number': 1065}	0.7394	0.7988	0.7680	0.8069
0.2647	14.0	140	0.7052	{'precision': 0.7184035476718403, 'recall': 0.8009888751545118, 'f1': 0.7574517825832847, 'number': 809}	{'precision': 0.3282442748091603, 'recall': 0.36134453781512604, 'f1': 0.344, 'number': 119}	{'precision': 0.7872340425531915, 'recall': 0.8338028169014085, 'f1': 0.8098495212038304, 'number': 1065}	0.7307	0.7923	0.7602	0.8076
0.265	15.0	150	0.7033	{'precision': 0.7135016465422612, 'recall': 0.8034610630407911, 'f1': 0.7558139534883722, 'number': 809}	{'precision': 0.34959349593495936, 'recall': 0.36134453781512604, 'f1': 0.35537190082644626, 'number': 119}	{'precision': 0.7895204262877442, 'recall': 0.8347417840375587, 'f1': 0.8115015974440895, 'number': 1065}	0.7324	0.7938	0.7619	0.8096

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.20.3