working

This model is a fine-tuned version of pborchert/BusinessBERT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.3070

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0257	50	5.4950
No log	0.0513	100	5.2169
No log	0.0770	150	5.0495
No log	0.1026	200	4.9407
No log	0.1283	250	4.8442
No log	0.1539	300	4.7955
No log	0.1796	350	4.6896
No log	0.2052	400	4.6560
No log	0.2309	450	4.6657
5.1165	0.2565	500	4.5673
5.1165	0.2822	550	4.5526
5.1165	0.3079	600	4.5253
5.1165	0.3335	650	4.4882
5.1165	0.3592	700	4.4493
5.1165	0.3848	750	4.4193
5.1165	0.4105	800	4.3838
5.1165	0.4361	850	4.3737
5.1165	0.4618	900	4.3586
5.1165	0.4874	950	4.3631
4.5802	0.5131	1000	4.3204
4.5802	0.5387	1050	4.3004
4.5802	0.5644	1100	4.2959
4.5802	0.5900	1150	4.2635
4.5802	0.6157	1200	4.2373
4.5802	0.6414	1250	4.2351
4.5802	0.6670	1300	4.2160
4.5802	0.6927	1350	4.1734
4.5802	0.7183	1400	4.1644
4.5802	0.7440	1450	4.1413
4.3933	0.7696	1500	4.1446
4.3933	0.7953	1550	4.1304
4.3933	0.8209	1600	4.1446
4.3933	0.8466	1650	4.1190
4.3933	0.8722	1700	4.1191
4.3933	0.8979	1750	4.0745
4.3933	0.9236	1800	4.0714
4.3933	0.9492	1850	4.0761
4.3933	0.9749	1900	4.0438
4.3933	1.0005	1950	4.0607
4.2488	1.0262	2000	4.0196
4.2488	1.0518	2050	4.0295
4.2488	1.0775	2100	3.9847
4.2488	1.1031	2150	3.9741
4.2488	1.1288	2200	3.9577
4.2488	1.1544	2250	3.9609
4.2488	1.1801	2300	3.9989
4.2488	1.2057	2350	3.9628
4.2488	1.2314	2400	3.9308
4.2488	1.2571	2450	3.9223
4.0985	1.2827	2500	3.9455
4.0985	1.3084	2550	3.9289
4.0985	1.3340	2600	3.9327
4.0985	1.3597	2650	3.8934
4.0985	1.3853	2700	3.8884
4.0985	1.4110	2750	3.9077
4.0985	1.4366	2800	3.8825
4.0985	1.4623	2850	3.8765
4.0985	1.4879	2900	3.8477
4.0985	1.5136	2950	3.8432
4.009	1.5393	3000	3.8259
4.009	1.5649	3050	3.8392
4.009	1.5906	3100	3.8668
4.009	1.6162	3150	3.8354
4.009	1.6419	3200	3.8150
4.009	1.6675	3250	3.8325
4.009	1.6932	3300	3.8268
4.009	1.7188	3350	3.8147
4.009	1.7445	3400	3.7826
4.009	1.7701	3450	3.7822
3.9676	1.7958	3500	3.7950
3.9676	1.8214	3550	3.7839
3.9676	1.8471	3600	3.7646
3.9676	1.8728	3650	3.7747
3.9676	1.8984	3700	3.7583
3.9676	1.9241	3750	3.7438
3.9676	1.9497	3800	3.7603
3.9676	1.9754	3850	3.7299
3.9676	2.0010	3900	3.7391
3.9676	2.0267	3950	3.7117
3.8702	2.0523	4000	3.7592
3.8702	2.0780	4050	3.7178
3.8702	2.1036	4100	3.7501
3.8702	2.1293	4150	3.7182
3.8702	2.1550	4200	3.7127
3.8702	2.1806	4250	3.7032
3.8702	2.2063	4300	3.7028
3.8702	2.2319	4350	3.6726
3.8702	2.2576	4400	3.6591
3.8702	2.2832	4450	3.6744
3.8324	2.3089	4500	3.6995
3.8324	2.3345	4550	3.6777
3.8324	2.3602	4600	3.7105
3.8324	2.3858	4650	3.6624
3.8324	2.4115	4700	3.6701
3.8324	2.4371	4750	3.6298
3.8324	2.4628	4800	3.6582
3.8324	2.4885	4850	3.6668
3.8324	2.5141	4900	3.6393
3.8324	2.5398	4950	3.6194
3.783	2.5654	5000	3.6175
3.783	2.5911	5050	3.6144
3.783	2.6167	5100	3.6148
3.783	2.6424	5150	3.5929
3.783	2.6680	5200	3.6180
3.783	2.6937	5250	3.6065
3.783	2.7193	5300	3.6218
3.783	2.7450	5350	3.6123
3.783	2.7707	5400	3.6084
3.783	2.7963	5450	3.5988
3.7504	2.8220	5500	3.5943
3.7504	2.8476	5550	3.5656
3.7504	2.8733	5600	3.6005
3.7504	2.8989	5650	3.5971
3.7504	2.9246	5700	3.5795
3.7504	2.9502	5750	3.5698
3.7504	2.9759	5800	3.5793
3.7504	3.0015	5850	3.5790
3.7504	3.0272	5900	3.5504
3.7504	3.0528	5950	3.5628
3.6774	3.0785	6000	3.5494
3.6774	3.1042	6050	3.5776
3.6774	3.1298	6100	3.5380
3.6774	3.1555	6150	3.5350
3.6774	3.1811	6200	3.5347
3.6774	3.2068	6250	3.5553
3.6774	3.2324	6300	3.5684
3.6774	3.2581	6350	3.5194
3.6774	3.2837	6400	3.5157
3.6774	3.3094	6450	3.5307
3.643	3.3350	6500	3.5300
3.643	3.3607	6550	3.5182
3.643	3.3864	6600	3.5084
3.643	3.4120	6650	3.5237
3.643	3.4377	6700	3.5138
3.643	3.4633	6750	3.4951
3.643	3.4890	6800	3.4666
3.643	3.5146	6850	3.5216
3.643	3.5403	6900	3.5231
3.643	3.5659	6950	3.4880
3.6139	3.5916	7000	3.5065
3.6139	3.6172	7050	3.4963
3.6139	3.6429	7100	3.4861
3.6139	3.6685	7150	3.4852
3.6139	3.6942	7200	3.4753
3.6139	3.7199	7250	3.4783
3.6139	3.7455	7300	3.4762
3.6139	3.7712	7350	3.4914
3.6139	3.7968	7400	3.4560
3.6139	3.8225	7450	3.4587
3.5909	3.8481	7500	3.4825
3.5909	3.8738	7550	3.4616
3.5909	3.8994	7600	3.4469
3.5909	3.9251	7650	3.4609
3.5909	3.9507	7700	3.4284
3.5909	3.9764	7750	3.4680
3.5909	4.0021	7800	3.4610
3.5909	4.0277	7850	3.4718
3.5909	4.0534	7900	3.4455
3.5909	4.0790	7950	3.4548
3.539	4.1047	8000	3.4461
3.539	4.1303	8050	3.4435
3.539	4.1560	8100	3.4412
3.539	4.1816	8150	3.4331
3.539	4.2073	8200	3.4258
3.539	4.2329	8250	3.4367
3.539	4.2586	8300	3.4259
3.539	4.2842	8350	3.4338
3.539	4.3099	8400	3.4274
3.539	4.3356	8450	3.4556
3.5276	4.3612	8500	3.4288
3.5276	4.3869	8550	3.4279
3.5276	4.4125	8600	3.4191
3.5276	4.4382	8650	3.3882
3.5276	4.4638	8700	3.4083
3.5276	4.4895	8750	3.4005
3.5276	4.5151	8800	3.3910
3.5276	4.5408	8850	3.4416
3.5276	4.5664	8900	3.4154
3.5276	4.5921	8950	3.4116
3.4812	4.6178	9000	3.3965
3.4812	4.6434	9050	3.3789
3.4812	4.6691	9100	3.3688
3.4812	4.6947	9150	3.3994
3.4812	4.7204	9200	3.3970
3.4812	4.7460	9250	3.3965
3.4812	4.7717	9300	3.4146
3.4812	4.7973	9350	3.3628
3.4812	4.8230	9400	3.4125
3.4812	4.8486	9450	3.3942
3.5039	4.8743	9500	3.3965
3.5039	4.8999	9550	3.3620
3.5039	4.9256	9600	3.4002
3.5039	4.9513	9650	3.3658
3.5039	4.9769	9700	3.3547
3.5039	5.0026	9750	3.3852
3.5039	5.0282	9800	3.3811
3.5039	5.0539	9850	3.3980
3.5039	5.0795	9900	3.3577
3.5039	5.1052	9950	3.3668
3.4764	5.1308	10000	3.3734
3.4764	5.1565	10050	3.3895
3.4764	5.1821	10100	3.3553
3.4764	5.2078	10150	3.3640
3.4764	5.2335	10200	3.3691
3.4764	5.2591	10250	3.3724
3.4764	5.2848	10300	3.3703
3.4764	5.3104	10350	3.3383
3.4764	5.3361	10400	3.3653
3.4764	5.3617	10450	3.3709
3.4419	5.3874	10500	3.3848
3.4419	5.4130	10550	3.3522
3.4419	5.4387	10600	3.3370
3.4419	5.4643	10650	3.3667
3.4419	5.4900	10700	3.3621
3.4419	5.5156	10750	3.3706
3.4419	5.5413	10800	3.3339
3.4419	5.5670	10850	3.3561
3.4419	5.5926	10900	3.3574
3.4419	5.6183	10950	3.3452
3.4277	5.6439	11000	3.3431
3.4277	5.6696	11050	3.3331
3.4277	5.6952	11100	3.3397
3.4277	5.7209	11150	3.3414
3.4277	5.7465	11200	3.3307
3.4277	5.7722	11250	3.3241
3.4277	5.7978	11300	3.3572
3.4277	5.8235	11350	3.3402
3.4277	5.8492	11400	3.3145
3.4277	5.8748	11450	3.3312
3.4197	5.9005	11500	3.3520
3.4197	5.9261	11550	3.3387
3.4197	5.9518	11600	3.3213
3.4197	5.9774	11650	3.3300
3.4197	6.0031	11700	3.3201
3.4197	6.0287	11750	3.3075
3.4197	6.0544	11800	3.3329
3.4197	6.0800	11850	3.3202
3.4197	6.1057	11900	3.3527
3.4197	6.1313	11950	3.3377
3.3882	6.1570	12000	3.3125
3.3882	6.1827	12050	3.3278
3.3882	6.2083	12100	3.3224
3.3882	6.2340	12150	3.3224
3.3882	6.2596	12200	3.3093
3.3882	6.2853	12250	3.3184
3.3882	6.3109	12300	3.3520
3.3882	6.3366	12350	3.3085
3.3882	6.3622	12400	3.2921
3.3882	6.3879	12450	3.3209
3.3896	6.4135	12500	3.3323
3.3896	6.4392	12550	3.3098
3.3896	6.4649	12600	3.3157
3.3896	6.4905	12650	3.3032
3.3896	6.5162	12700	3.3247
3.3896	6.5418	12750	3.3171
3.3896	6.5675	12800	3.2847
3.3896	6.5931	12850	3.3035
3.3896	6.6188	12900	3.2980
3.3896	6.6444	12950	3.3195
3.3973	6.6701	13000	3.3071
3.3973	6.6957	13050	3.3037
3.3973	6.7214	13100	3.3250
3.3973	6.7470	13150	3.3324
3.3973	6.7727	13200	3.3228
3.3973	6.7984	13250	3.2844
3.3973	6.8240	13300	3.3302
3.3973	6.8497	13350	3.3019
3.3973	6.8753	13400	3.3113
3.3973	6.9010	13450	3.3134
3.3858	6.9266	13500	3.2894
3.3858	6.9523	13550	3.3094
3.3858	6.9779	13600	3.3070

Framework versions

• Transformers 4.49.0.dev0
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0