Pass your compute_metrics function to KerasMetricCallback, and use the PushToHubCallback to upload the model: from transformers.keras_callbacks import KerasMetricCallback, PushToHubCallback metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_eval_dataset) push_to_hub_callback = PushToHubCallback( output_dir="food_classifier", tokenizer=image_processor, save_strategy="no", ) callbacks = [metric_callback, push_to_hub_callback] Finally, you are ready to train your model!