Pass your compute_metrics function to [~transformers.KerasMetricCallback]:

from transformers.keras_callbacks import KerasMetricCallback
metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_validation_set)

Specify where to push your model and tokenizer in the [~transformers.PushToHubCallback]:

from transformers.keras_callbacks import PushToHubCallback
push_to_hub_callback = PushToHubCallback(
     output_dir="my_awesome_model",
     tokenizer=tokenizer,
 )

Then bundle your callbacks together:

callbacks = [metric_callback, push_to_hub_callback]

Finally, you're ready to start training your model!