|
|
--- |
|
|
library_name: transformers |
|
|
language: |
|
|
- eu |
|
|
license: apache-2.0 |
|
|
base_model: openai/whisper-large |
|
|
tags: |
|
|
- whisper-event |
|
|
- generated_from_trainer |
|
|
datasets: |
|
|
- mozilla-foundation/common_voice_17_0 |
|
|
metrics: |
|
|
- wer |
|
|
model-index: |
|
|
- name: Whisper Large Basque |
|
|
results: |
|
|
- task: |
|
|
name: Automatic Speech Recognition |
|
|
type: automatic-speech-recognition |
|
|
dataset: |
|
|
name: mozilla-foundation/common_voice_17_0 eu |
|
|
type: mozilla-foundation/common_voice_17_0 |
|
|
config: eu |
|
|
split: test |
|
|
args: eu |
|
|
metrics: |
|
|
- name: Wer |
|
|
type: wer |
|
|
value: 7.819084612847668 |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# Whisper Large Basque |
|
|
|
|
|
This model is a fine-tuned version of [openai/whisper-large](https://huggingface.co/openai/whisper-large) on the mozilla-foundation/common_voice_17_0 eu dataset. |
|
|
It achieves the following results on the evaluation set: |
|
|
- Loss: 0.3149 |
|
|
- Wer: 7.8191 |
|
|
|
|
|
## Model description |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training and evaluation data |
|
|
|
|
|
More information needed |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 3.75e-05 |
|
|
- train_batch_size: 32 |
|
|
- eval_batch_size: 16 |
|
|
- seed: 42 |
|
|
- gradient_accumulation_steps: 2 |
|
|
- total_train_batch_size: 64 |
|
|
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
|
- lr_scheduler_type: linear |
|
|
- lr_scheduler_warmup_steps: 500 |
|
|
- training_steps: 40000 |
|
|
- mixed_precision_training: Native AMP |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|
|:-------------:|:-------:|:-----:|:---------------:|:-------:| |
|
|
| 0.0767 | 2.3474 | 1000 | 0.1933 | 11.7382 | |
|
|
| 0.0404 | 4.6948 | 2000 | 0.2059 | 10.3485 | |
|
|
| 0.0188 | 7.0423 | 3000 | 0.2262 | 9.8987 | |
|
|
| 0.0121 | 9.3897 | 4000 | 0.2340 | 9.7997 | |
|
|
| 0.0123 | 11.7371 | 5000 | 0.2298 | 9.4910 | |
|
|
| 0.0061 | 14.0845 | 6000 | 0.2398 | 9.3847 | |
|
|
| 0.0094 | 16.4319 | 7000 | 0.2416 | 9.7750 | |
|
|
| 0.0068 | 18.7793 | 8000 | 0.2533 | 9.6055 | |
|
|
| 0.0061 | 21.1268 | 9000 | 0.2569 | 9.2995 | |
|
|
| 0.0054 | 23.4742 | 10000 | 0.2646 | 9.5386 | |
|
|
| 0.0063 | 25.8216 | 11000 | 0.2684 | 9.7686 | |
|
|
| 0.0052 | 28.1690 | 12000 | 0.2600 | 9.2666 | |
|
|
| 0.0042 | 30.5164 | 13000 | 0.2763 | 9.4654 | |
|
|
| 0.0039 | 32.8638 | 14000 | 0.2705 | 8.9981 | |
|
|
| 0.0021 | 35.2113 | 15000 | 0.2640 | 9.0513 | |
|
|
| 0.0026 | 37.5587 | 16000 | 0.2699 | 9.3435 | |
|
|
| 0.0029 | 39.9061 | 17000 | 0.2838 | 9.2033 | |
|
|
| 0.0016 | 42.2535 | 18000 | 0.2784 | 9.2794 | |
|
|
| 0.0033 | 44.6009 | 19000 | 0.2756 | 9.0558 | |
|
|
| 0.0023 | 46.9484 | 20000 | 0.2804 | 9.6110 | |
|
|
| 0.0024 | 49.2958 | 21000 | 0.2809 | 9.0439 | |
|
|
| 0.001 | 51.6432 | 22000 | 0.2824 | 8.6940 | |
|
|
| 0.0028 | 53.9906 | 23000 | 0.2890 | 9.3829 | |
|
|
| 0.0007 | 56.3380 | 24000 | 0.2807 | 8.7398 | |
|
|
| 0.0004 | 58.6854 | 25000 | 0.2881 | 8.6738 | |
|
|
| 0.0007 | 61.0329 | 26000 | 0.2888 | 8.9175 | |
|
|
| 0.0008 | 63.3803 | 27000 | 0.2937 | 8.9578 | |
|
|
| 0.0003 | 65.7277 | 28000 | 0.2858 | 8.6299 | |
|
|
| 0.0008 | 68.0751 | 29000 | 0.2877 | 8.8598 | |
|
|
| 0.0006 | 70.4225 | 30000 | 0.2930 | 8.6308 | |
|
|
| 0.0002 | 72.7700 | 31000 | 0.2852 | 8.4338 | |
|
|
| 0.0 | 75.1174 | 32000 | 0.2936 | 8.4530 | |
|
|
| 0.0 | 77.4648 | 33000 | 0.2968 | 8.2002 | |
|
|
| 0.0 | 79.8122 | 34000 | 0.3008 | 8.1086 | |
|
|
| 0.0 | 82.1596 | 35000 | 0.3043 | 8.0243 | |
|
|
| 0.0 | 84.5070 | 36000 | 0.3074 | 7.9730 | |
|
|
| 0.0 | 86.8545 | 37000 | 0.3101 | 7.9116 | |
|
|
| 0.0 | 89.2019 | 38000 | 0.3124 | 7.8502 | |
|
|
| 0.0 | 91.5493 | 39000 | 0.3141 | 7.8282 | |
|
|
| 0.0 | 93.8967 | 40000 | 0.3149 | 7.8191 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.52.3 |
|
|
- Pytorch 2.6.0+cu124 |
|
|
- Datasets 3.6.0 |
|
|
- Tokenizers 0.21.1 |
|
|
|
