They both rely on | |
[~tokenization_utils_base.PreTrainedTokenizerBase] that contains the common methods, and | |
[~tokenization_utils_base.SpecialTokensMixin]. |
They both rely on | |
[~tokenization_utils_base.PreTrainedTokenizerBase] that contains the common methods, and | |
[~tokenization_utils_base.SpecialTokensMixin]. |