roberta-basemhr2004-atomic.anion.train.no1e-06-128

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 512
• eval_batch_size: 1024
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
0.6094	1.0	576	0.5127
0.5113	2.0	1152	0.4806
0.488	3.0	1728	0.4670
0.4759	4.0	2304	0.4560
0.4641	5.0	2880	0.4496
0.4572	6.0	3456	0.4432
0.4512	7.0	4032	0.4390
0.4452	8.0	4608	0.4343
0.4407	9.0	5184	0.4338
0.4387	10.0	5760	0.4299
0.4333	11.0	6336	0.4272
0.4332	12.0	6912	0.4242
0.4257	13.0	7488	0.4244
0.4245	14.0	8064	0.4229
0.4207	15.0	8640	0.4212
0.4209	16.0	9216	0.4184
0.4167	17.0	9792	0.4185
0.417	18.0	10368	0.4179
0.4163	19.0	10944	0.4178
0.4119	20.0	11520	0.4167
0.4089	21.0	12096	0.4159
0.4125	22.0	12672	0.4153
0.4097	23.0	13248	0.4143
0.4092	24.0	13824	0.4141
0.4073	25.0	14400	0.4133
0.408	26.0	14976	0.4135
0.4054	27.0	15552	0.4135
0.4068	28.0	16128	0.4131
0.4043	29.0	16704	0.4133
0.4069	30.0	17280	0.4132

Framework versions

• Transformers 4.51.2
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1