Se124M100KInfPrompt_WT

This model is a fine-tuned version of gpt2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7371

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
3.2505	0.0082	20	2.8871
3.1482	0.0164	40	2.8898
3.1815	0.0246	60	2.8780
3.1657	0.0327	80	2.8574
3.0926	0.0409	100	2.8249
3.1184	0.0491	120	2.7654
3.0128	0.0573	140	2.7067
2.9348	0.0655	160	2.6351
2.7728	0.0737	180	2.5450
2.6372	0.0819	200	2.4318
2.4966	0.0901	220	2.2950
2.3591	0.0982	240	2.1465
2.2302	0.1064	260	2.0135
2.0753	0.1146	280	1.8699
1.9052	0.1228	300	1.7487
1.8	0.1310	320	1.6347
1.7122	0.1392	340	1.5290
1.6217	0.1474	360	1.4386
1.5754	0.1555	380	1.3520
1.4438	0.1637	400	1.2721
1.4155	0.1719	420	1.2061
1.3491	0.1801	440	1.1527
1.2966	0.1883	460	1.1089
1.2319	0.1965	480	1.0730
1.2031	0.2047	500	1.0470
1.1872	0.2129	520	1.0232
1.1362	0.2210	540	1.0026
1.13	0.2292	560	0.9844
1.0864	0.2374	580	0.9677
1.0712	0.2456	600	0.9563
1.0732	0.2538	620	0.9418
1.0519	0.2620	640	0.9327
1.0337	0.2702	660	0.9218
1.0408	0.2783	680	0.9093
1.004	0.2865	700	0.9030
0.9896	0.2947	720	0.8942
0.9668	0.3029	740	0.8870
0.9539	0.3111	760	0.8814
0.953	0.3193	780	0.8736
0.9388	0.3275	800	0.8696
0.9497	0.3357	820	0.8647
0.9309	0.3438	840	0.8619
0.9326	0.3520	860	0.8568
0.9272	0.3602	880	0.8519
0.9355	0.3684	900	0.8498
0.9147	0.3766	920	0.8460
0.9189	0.3848	940	0.8431
0.9061	0.3930	960	0.8394
0.9121	0.4011	980	0.8376
0.9007	0.4093	1000	0.8373
0.8897	0.4175	1020	0.8344
0.9037	0.4257	1040	0.8326
0.8987	0.4339	1060	0.8282
0.8968	0.4421	1080	0.8260
0.8906	0.4503	1100	0.8258
0.8915	0.4585	1120	0.8231
0.8911	0.4666	1140	0.8194
0.892	0.4748	1160	0.8171
0.8725	0.4830	1180	0.8169
0.8732	0.4912	1200	0.8168
0.8752	0.4994	1220	0.8154
0.8679	0.5076	1240	0.8147
0.8519	0.5158	1260	0.8139
0.8621	0.5239	1280	0.8092
0.8516	0.5321	1300	0.8093
0.8588	0.5403	1320	0.8070
0.8777	0.5485	1340	0.8080
0.8517	0.5567	1360	0.8050
0.8572	0.5649	1380	0.8032
0.8408	0.5731	1400	0.8052
0.8509	0.5813	1420	0.8042
0.8478	0.5894	1440	0.8039
0.8422	0.5976	1460	0.7995
0.8348	0.6058	1480	0.7999
0.8328	0.6140	1500	0.7998
0.8358	0.6222	1520	0.7988
0.825	0.6304	1540	0.7978
0.8342	0.6386	1560	0.7975
0.839	0.6467	1580	0.7963
0.8294	0.6549	1600	0.7954
0.8523	0.6631	1620	0.7958
0.8294	0.6713	1640	0.7922
0.8279	0.6795	1660	0.7939
0.8094	0.6877	1680	0.7951
0.8388	0.6959	1700	0.7914
0.8256	0.7041	1720	0.7907
0.8303	0.7122	1740	0.7906
0.8196	0.7204	1760	0.7901
0.8139	0.7286	1780	0.7891
0.8269	0.7368	1800	0.7880
0.8265	0.7450	1820	0.7868
0.835	0.7532	1840	0.7838
0.8354	0.7614	1860	0.7852
0.8209	0.7695	1880	0.7842
0.8135	0.7777	1900	0.7823
0.8207	0.7859	1920	0.7823
0.8251	0.7941	1940	0.7820
0.8063	0.8023	1960	0.7822
0.829	0.8105	1980	0.7800
0.8163	0.8187	2000	0.7815
0.8266	0.8269	2020	0.7792
0.835	0.8350	2040	0.7786
0.8102	0.8432	2060	0.7779
0.8296	0.8514	2080	0.7771
0.7994	0.8596	2100	0.7776
0.8085	0.8678	2120	0.7744
0.8123	0.8760	2140	0.7738
0.811	0.8842	2160	0.7748
0.8232	0.8923	2180	0.7738
0.8053	0.9005	2200	0.7740
0.82	0.9087	2220	0.7719
0.8112	0.9169	2240	0.7726
0.832	0.9251	2260	0.7712
0.8147	0.9333	2280	0.7711
0.7964	0.9415	2300	0.7715
0.8108	0.9497	2320	0.7688
0.8086	0.9578	2340	0.7703
0.7982	0.9660	2360	0.7698
0.8012	0.9742	2380	0.7681
0.8217	0.9824	2400	0.7668
0.8001	0.9906	2420	0.7677
0.8066	0.9988	2440	0.7676
0.7948	1.0070	2460	0.7648
0.8126	1.0151	2480	0.7648
0.8062	1.0233	2500	0.7639
0.8094	1.0315	2520	0.7665
0.7977	1.0397	2540	0.7648
0.8154	1.0479	2560	0.7635
0.7989	1.0561	2580	0.7645
0.7976	1.0643	2600	0.7642
0.8038	1.0725	2620	0.7624
0.7932	1.0806	2640	0.7615
0.8001	1.0888	2660	0.7625
0.8049	1.0970	2680	0.7617
0.7959	1.1052	2700	0.7601
0.8094	1.1134	2720	0.7623
0.7935	1.1216	2740	0.7619
0.7844	1.1298	2760	0.7620
0.7842	1.1379	2780	0.7605
0.789	1.1461	2800	0.7626
0.7963	1.1543	2820	0.7606
0.7908	1.1625	2840	0.7578
0.7906	1.1707	2860	0.7588
0.7819	1.1789	2880	0.7611
0.8136	1.1871	2900	0.7594
0.8006	1.1953	2920	0.7598
0.8006	1.2034	2940	0.7585
0.7933	1.2116	2960	0.7571
0.7872	1.2198	2980	0.7595
0.7915	1.2280	3000	0.7560
0.7963	1.2362	3020	0.7557
0.7911	1.2444	3040	0.7577
0.788	1.2526	3060	0.7562
0.7883	1.2607	3080	0.7558
0.7901	1.2689	3100	0.7555
0.7839	1.2771	3120	0.7551
0.8046	1.2853	3140	0.7560
0.7944	1.2935	3160	0.7547
0.7909	1.3017	3180	0.7547
0.7867	1.3099	3200	0.7554
0.7877	1.3181	3220	0.7537
0.781	1.3262	3240	0.7531
0.7902	1.3344	3260	0.7531
0.788	1.3426	3280	0.7555
0.7906	1.3508	3300	0.7555
0.7856	1.3590	3320	0.7544
0.7877	1.3672	3340	0.7532
0.7925	1.3754	3360	0.7525
0.7841	1.3835	3380	0.7534
0.799	1.3917	3400	0.7520
0.7876	1.3999	3420	0.7500
0.7769	1.4081	3440	0.7510
0.8041	1.4163	3460	0.7500
0.7893	1.4245	3480	0.7526
0.7774	1.4327	3500	0.7503
0.782	1.4409	3520	0.7501
0.7824	1.4490	3540	0.7510
0.7813	1.4572	3560	0.7505
0.7919	1.4654	3580	0.7513
0.7801	1.4736	3600	0.7505
0.7751	1.4818	3620	0.7502
0.7723	1.4900	3640	0.7488
0.7841	1.4982	3660	0.7484
0.7938	1.5063	3680	0.7490
0.7888	1.5145	3700	0.7496
0.7831	1.5227	3720	0.7487
0.7881	1.5309	3740	0.7491
0.7933	1.5391	3760	0.7464
0.781	1.5473	3780	0.7491
0.7885	1.5555	3800	0.7474
0.7856	1.5637	3820	0.7475
0.7871	1.5718	3840	0.7471
0.7829	1.5800	3860	0.7464
0.8159	1.5882	3880	0.7464
0.7836	1.5964	3900	0.7466
0.7825	1.6046	3920	0.7472
0.7689	1.6128	3940	0.7466
0.776	1.6210	3960	0.7476
0.7718	1.6291	3980	0.7461
0.7905	1.6373	4000	0.7462
0.7776	1.6455	4020	0.7475
0.7743	1.6537	4040	0.7462
0.7778	1.6619	4060	0.7455
0.7928	1.6701	4080	0.7449
0.8031	1.6783	4100	0.7451
0.7845	1.6865	4120	0.7440
0.7763	1.6946	4140	0.7453
0.7841	1.7028	4160	0.7455
0.7814	1.7110	4180	0.7450
0.7843	1.7192	4200	0.7441
0.7733	1.7274	4220	0.7449
0.7779	1.7356	4240	0.7437
0.7855	1.7438	4260	0.7448
0.7775	1.7519	4280	0.7443
0.7802	1.7601	4300	0.7432
0.783	1.7683	4320	0.7431
0.7753	1.7765	4340	0.7441
0.7772	1.7847	4360	0.7433
0.7813	1.7929	4380	0.7432
0.7817	1.8011	4400	0.7423
0.7769	1.8093	4420	0.7426
0.7843	1.8174	4440	0.7428
0.7719	1.8256	4460	0.7428
0.7872	1.8338	4480	0.7427
0.7741	1.8420	4500	0.7421
0.7683	1.8502	4520	0.7422
0.7844	1.8584	4540	0.7433
0.7705	1.8666	4560	0.7425
0.7838	1.8747	4580	0.7427
0.7822	1.8829	4600	0.7422
0.7867	1.8911	4620	0.7415
0.7742	1.8993	4640	0.7428
0.7683	1.9075	4660	0.7420
0.7706	1.9157	4680	0.7413
0.7804	1.9239	4700	0.7420
0.7951	1.9321	4720	0.7417
0.7686	1.9402	4740	0.7411
0.7798	1.9484	4760	0.7400
0.7885	1.9566	4780	0.7402
0.7757	1.9648	4800	0.7408
0.7783	1.9730	4820	0.7408
0.7679	1.9812	4840	0.7404
0.7767	1.9894	4860	0.7409
0.7676	1.9975	4880	0.7415
0.7548	2.0057	4900	0.7410
0.7687	2.0139	4920	0.7414
0.7895	2.0221	4940	0.7403
0.7826	2.0303	4960	0.7403
0.7675	2.0385	4980	0.7419
0.7714	2.0467	5000	0.7401
0.7686	2.0549	5020	0.7417
0.7645	2.0630	5040	0.7408
0.7792	2.0712	5060	0.7403
0.77	2.0794	5080	0.7396
0.7752	2.0876	5100	0.7390
0.7797	2.0958	5120	0.7398
0.7785	2.1040	5140	0.7401
0.7727	2.1122	5160	0.7403
0.7748	2.1203	5180	0.7395
0.7657	2.1285	5200	0.7396
0.7709	2.1367	5220	0.7405
0.7947	2.1449	5240	0.7394
0.7758	2.1531	5260	0.7396
0.779	2.1613	5280	0.7397
0.7727	2.1695	5300	0.7395
0.7841	2.1777	5320	0.7394
0.7809	2.1858	5340	0.7391
0.7722	2.1940	5360	0.7398
0.7703	2.2022	5380	0.7391
0.7845	2.2104	5400	0.7390
0.7691	2.2186	5420	0.7392
0.7781	2.2268	5440	0.7397
0.7719	2.2350	5460	0.7382
0.7829	2.2431	5480	0.7383
0.7839	2.2513	5500	0.7391
0.7666	2.2595	5520	0.7384
0.782	2.2677	5540	0.7390
0.7773	2.2759	5560	0.7389
0.7844	2.2841	5580	0.7385
0.7522	2.2923	5600	0.7388
0.7645	2.3005	5620	0.7394
0.7921	2.3086	5640	0.7377
0.7716	2.3168	5660	0.7378
0.7699	2.3250	5680	0.7384
0.7812	2.3332	5700	0.7385
0.7853	2.3414	5720	0.7387
0.7898	2.3496	5740	0.7384
0.7727	2.3578	5760	0.7376
0.7752	2.3659	5780	0.7374
0.7723	2.3741	5800	0.7379
0.7611	2.3823	5820	0.7383
0.7733	2.3905	5840	0.7380
0.7733	2.3987	5860	0.7382
0.7723	2.4069	5880	0.7375
0.777	2.4151	5900	0.7379
0.7733	2.4233	5920	0.7379
0.7788	2.4314	5940	0.7379
0.769	2.4396	5960	0.7371
0.7832	2.4478	5980	0.7385
0.763	2.4560	6000	0.7380
0.7807	2.4642	6020	0.7380
0.7875	2.4724	6040	0.7374
0.7711	2.4806	6060	0.7376
0.7774	2.4887	6080	0.7384
0.7843	2.4969	6100	0.7377
0.7717	2.5051	6120	0.7375
0.7611	2.5133	6140	0.7372
0.7804	2.5215	6160	0.7373
0.7818	2.5297	6180	0.7377
0.7635	2.5379	6200	0.7373
0.7699	2.5460	6220	0.7381
0.7751	2.5542	6240	0.7378
0.7729	2.5624	6260	0.7384
0.7645	2.5706	6280	0.7375
0.7653	2.5788	6300	0.7381
0.7776	2.5870	6320	0.7383
0.7812	2.5952	6340	0.7376
0.7597	2.6034	6360	0.7374
0.7627	2.6115	6380	0.7370
0.7722	2.6197	6400	0.7378
0.7832	2.6279	6420	0.7373
0.7723	2.6361	6440	0.7370
0.7655	2.6443	6460	0.7372
0.7825	2.6525	6480	0.7373
0.7677	2.6607	6500	0.7377
0.7728	2.6688	6520	0.7376
0.779	2.6770	6540	0.7370
0.7693	2.6852	6560	0.7369
0.7601	2.6934	6580	0.7374
0.7768	2.7016	6600	0.7373
0.7792	2.7098	6620	0.7373
0.7678	2.7180	6640	0.7374
0.7822	2.7262	6660	0.7376
0.7774	2.7343	6680	0.7371
0.7689	2.7425	6700	0.7373
0.7681	2.7507	6720	0.7373
0.7665	2.7589	6740	0.7374
0.7718	2.7671	6760	0.7372
0.7708	2.7753	6780	0.7375
0.7703	2.7835	6800	0.7374
0.7611	2.7916	6820	0.7372
0.7702	2.7998	6840	0.7375
0.7736	2.8080	6860	0.7376
0.7767	2.8162	6880	0.7371
0.7913	2.8244	6900	0.7369
0.7761	2.8326	6920	0.7375
0.7805	2.8408	6940	0.7377
0.7715	2.8490	6960	0.7374
0.77	2.8571	6980	0.7377
0.7688	2.8653	7000	0.7377
0.7721	2.8735	7020	0.7374
0.7834	2.8817	7040	0.7371
0.7747	2.8899	7060	0.7377
0.7817	2.8981	7080	0.7375
0.773	2.9063	7100	0.7371
0.7694	2.9144	7120	0.7377
0.7961	2.9226	7140	0.7374
0.7653	2.9308	7160	0.7377
0.7582	2.9390	7180	0.7375
0.775	2.9472	7200	0.7375
0.7741	2.9554	7220	0.7373
0.7789	2.9636	7240	0.7382
0.7632	2.9718	7260	0.7373
0.777	2.9799	7280	0.7370
0.7652	2.9881	7300	0.7370
0.7671	2.9963	7320	0.7371

Framework versions

• PEFT 0.15.1
• Transformers 4.51.3
• Pytorch 2.6.0+cu118
• Datasets 3.5.0
• Tokenizers 0.21.1