jckim/checkpoints

Whisper Turbo Multilingual (ko, ja, zh, en)

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the custom_multilingual dataset. It achieves the following results on the evaluation set:

• Loss: 0.3860
• Wer: 15.9354

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3244	1.0971	500	0.3557	19.6171
0.2701	2.1942	1000	0.3490	19.5934
0.224	3.2913	1500	0.3503	17.5891
0.2132	4.3884	2000	0.3518	16.9210
0.1865	5.4855	2500	0.3571	16.2908
0.1661	6.5826	3000	0.3652	16.0491
0.1467	7.6796	3500	0.3692	16.4471
0.136	8.7767	4000	0.3762	15.9496
0.1229	9.8738	4500	0.3816	15.8453
0.1146	10.9709	5000	0.3860	15.9354

Framework versions

• PEFT 0.15.2.dev0
• Transformers 4.46.3
• Pytorch 2.3.1+cu121
• Datasets 3.0.0
• Tokenizers 0.20.3