from transformers import DefaultDataCollator data_collator = DefaultDataCollator() To avoid overfitting and to make the model more robust, add some data augmentation to the training part of the dataset.