layoutlm-funsd2

This model is a fine-tuned version of srinidhireddy1604/layoutlm-funsd on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.8155
• Answer: {'precision': 0.7465437788018433, 'recall': 0.8009888751545118, 'f1': 0.7728085867620751, 'number': 809}
• Header: {'precision': 0.3597122302158273, 'recall': 0.42016806722689076, 'f1': 0.38759689922480617, 'number': 119}
• Question: {'precision': 0.8012533572068039, 'recall': 0.8403755868544601, 'f1': 0.8203483043079742, 'number': 1065}
• Overall Precision: 0.75
• Overall Recall: 0.7993
• Overall F1: 0.7739
• Overall Accuracy: 0.8050

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.5269	1.0	5	0.7058	{'precision': 0.6515151515151515, 'recall': 0.7972805933250927, 'f1': 0.717065036131184, 'number': 809}	{'precision': 0.4032258064516129, 'recall': 0.21008403361344538, 'f1': 0.2762430939226519, 'number': 119}	{'precision': 0.7341659232827832, 'recall': 0.7727699530516432, 'f1': 0.7529734675205856, 'number': 1065}	0.6871	0.7491	0.7168	0.7949
0.4603	2.0	10	0.7202	{'precision': 0.6969339622641509, 'recall': 0.73053152039555, 'f1': 0.7133373566686784, 'number': 809}	{'precision': 0.2896551724137931, 'recall': 0.35294117647058826, 'f1': 0.31818181818181823, 'number': 119}	{'precision': 0.7457777777777778, 'recall': 0.787793427230047, 'f1': 0.7662100456621005, 'number': 1065}	0.6950	0.7386	0.7161	0.7921
0.4097	3.0	15	0.6983	{'precision': 0.7, 'recall': 0.7787391841779975, 'f1': 0.7372732592159158, 'number': 809}	{'precision': 0.30701754385964913, 'recall': 0.29411764705882354, 'f1': 0.30042918454935624, 'number': 119}	{'precision': 0.7397831526271893, 'recall': 0.8328638497652582, 'f1': 0.7835689045936396, 'number': 1065}	0.7013	0.7787	0.7380	0.8069
0.3584	4.0	20	0.7083	{'precision': 0.7253121452894438, 'recall': 0.7898640296662547, 'f1': 0.7562130177514793, 'number': 809}	{'precision': 0.3305785123966942, 'recall': 0.33613445378151263, 'f1': 0.33333333333333337, 'number': 119}	{'precision': 0.7732610659439928, 'recall': 0.8037558685446009, 'f1': 0.7882136279926335, 'number': 1065}	0.7278	0.7702	0.7484	0.8095
0.325	5.0	25	0.7201	{'precision': 0.6947483588621444, 'recall': 0.7849196538936959, 'f1': 0.7370864770748694, 'number': 809}	{'precision': 0.2962962962962963, 'recall': 0.33613445378151263, 'f1': 0.31496062992125984, 'number': 119}	{'precision': 0.7645021645021645, 'recall': 0.8291079812206573, 'f1': 0.7954954954954955, 'number': 1065}	0.7069	0.7817	0.7424	0.8031
0.292	6.0	30	0.7411	{'precision': 0.7247706422018348, 'recall': 0.7812113720642769, 'f1': 0.7519333729922665, 'number': 809}	{'precision': 0.36036036036036034, 'recall': 0.33613445378151263, 'f1': 0.34782608695652173, 'number': 119}	{'precision': 0.7846846846846847, 'recall': 0.8178403755868544, 'f1': 0.800919540229885, 'number': 1065}	0.7372	0.7742	0.7553	0.8057
0.2752	7.0	35	0.7645	{'precision': 0.7170022371364653, 'recall': 0.792336217552534, 'f1': 0.7527891955372871, 'number': 809}	{'precision': 0.3032258064516129, 'recall': 0.3949579831932773, 'f1': 0.34306569343065696, 'number': 119}	{'precision': 0.7769848349687779, 'recall': 0.8178403755868544, 'f1': 0.7968892955169259, 'number': 1065}	0.7184	0.7822	0.7490	0.7943
0.2508	8.0	40	0.7613	{'precision': 0.7331081081081081, 'recall': 0.8046971569839307, 'f1': 0.7672362993517973, 'number': 809}	{'precision': 0.34959349593495936, 'recall': 0.36134453781512604, 'f1': 0.35537190082644626, 'number': 119}	{'precision': 0.7905944986690329, 'recall': 0.8366197183098592, 'f1': 0.8129562043795621, 'number': 1065}	0.7413	0.7953	0.7674	0.8012
0.2305	9.0	45	0.7761	{'precision': 0.7379862700228833, 'recall': 0.7972805933250927, 'f1': 0.7664884135472371, 'number': 809}	{'precision': 0.3161290322580645, 'recall': 0.4117647058823529, 'f1': 0.35766423357664234, 'number': 119}	{'precision': 0.7846975088967971, 'recall': 0.828169014084507, 'f1': 0.8058474189127455, 'number': 1065}	0.7320	0.7908	0.7603	0.8018
0.2201	10.0	50	0.7905	{'precision': 0.7369614512471655, 'recall': 0.8034610630407911, 'f1': 0.768775872264932, 'number': 809}	{'precision': 0.3409090909090909, 'recall': 0.37815126050420167, 'f1': 0.3585657370517928, 'number': 119}	{'precision': 0.791814946619217, 'recall': 0.8356807511737089, 'f1': 0.8131566925536775, 'number': 1065}	0.7413	0.7953	0.7674	0.8033
0.2091	11.0	55	0.8025	{'precision': 0.7281879194630873, 'recall': 0.8046971569839307, 'f1': 0.7645331767469172, 'number': 809}	{'precision': 0.33783783783783783, 'recall': 0.42016806722689076, 'f1': 0.37453183520599254, 'number': 119}	{'precision': 0.8009009009009009, 'recall': 0.8347417840375587, 'f1': 0.8174712643678161, 'number': 1065}	0.7388	0.7978	0.7672	0.8014
0.197	12.0	60	0.8051	{'precision': 0.74230330672748, 'recall': 0.8046971569839307, 'f1': 0.7722419928825623, 'number': 809}	{'precision': 0.3401360544217687, 'recall': 0.42016806722689076, 'f1': 0.37593984962406013, 'number': 119}	{'precision': 0.8025247971145176, 'recall': 0.8356807511737089, 'f1': 0.8187672493100275, 'number': 1065}	0.7459	0.7983	0.7712	0.8035
0.1899	13.0	65	0.8079	{'precision': 0.7448275862068966, 'recall': 0.8009888751545118, 'f1': 0.7718880285884455, 'number': 809}	{'precision': 0.3671875, 'recall': 0.3949579831932773, 'f1': 0.3805668016194332, 'number': 119}	{'precision': 0.7996422182468694, 'recall': 0.8394366197183099, 'f1': 0.8190563444800733, 'number': 1065}	0.7509	0.7973	0.7734	0.8048
0.1839	14.0	70	0.8127	{'precision': 0.7442660550458715, 'recall': 0.8022249690976514, 'f1': 0.7721594289113624, 'number': 809}	{'precision': 0.36496350364963503, 'recall': 0.42016806722689076, 'f1': 0.390625, 'number': 119}	{'precision': 0.8019713261648745, 'recall': 0.8403755868544601, 'f1': 0.8207244383310408, 'number': 1065}	0.7501	0.7998	0.7742	0.8056
0.1821	15.0	75	0.8155	{'precision': 0.7465437788018433, 'recall': 0.8009888751545118, 'f1': 0.7728085867620751, 'number': 809}	{'precision': 0.3597122302158273, 'recall': 0.42016806722689076, 'f1': 0.38759689922480617, 'number': 119}	{'precision': 0.8012533572068039, 'recall': 0.8403755868544601, 'f1': 0.8203483043079742, 'number': 1065}	0.75	0.7993	0.7739	0.8050

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0