mT5

Overview
The mT5 model was presented in mT5: A massively multilingual pre-trained text-to-text transformer by Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya
Siddhant, Aditya Barua, Colin Raffel.
The abstract from the paper is the following:
The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain
state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a
multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. We detail
the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual
benchmarks. We also describe a simple technique to prevent "accidental translation" in the zero-shot setting, where a
generative model chooses to (partially) translate its prediction into the wrong language. All of the code and model
checkpoints used in this work are publicly available.
Note: mT5 was only pre-trained on mC4 excluding any supervised training.
Therefore, this model has to be fine-tuned before it is usable on a downstream task, unlike the original T5 model.
Since mT5 was pre-trained unsupervisedly, there's no real advantage to using a task prefix during single-task
fine-tuning. If you are doing multi-task fine-tuning, you should use a prefix.
Google has released the following variants:

google/mt5-small

google/mt5-base

google/mt5-large

google/mt5-xl

google/mt5-xxl.

This model was contributed by patrickvonplaten. The original code can be
found here.
Resources

Translation task guide
Summarization task guide

MT5Config
[[autodoc]] MT5Config
MT5Tokenizer
[[autodoc]] MT5Tokenizer
See [T5Tokenizer] for all details.
MT5TokenizerFast
[[autodoc]] MT5TokenizerFast
See [T5TokenizerFast] for all details.

MT5Model
[[autodoc]] MT5Model
MT5ForConditionalGeneration
[[autodoc]] MT5ForConditionalGeneration
MT5EncoderModel
[[autodoc]] MT5EncoderModel
MT5ForSequenceClassification
[[autodoc]] MT5ForSequenceClassification
MT5ForTokenClassification
[[autodoc]] MT5ForTokenClassification
MT5ForQuestionAnswering
[[autodoc]] MT5ForQuestionAnswering

TFMT5Model
[[autodoc]] TFMT5Model
TFMT5ForConditionalGeneration
[[autodoc]] TFMT5ForConditionalGeneration
TFMT5EncoderModel
[[autodoc]] TFMT5EncoderModel

FlaxMT5Model
[[autodoc]] FlaxMT5Model
FlaxMT5ForConditionalGeneration
[[autodoc]] FlaxMT5ForConditionalGeneration
FlaxMT5EncoderModel
[[autodoc]] FlaxMT5EncoderModel