| The base classes [PreTrainedTokenizer] and [PreTrainedTokenizerFast] | |
| implement the common methods for encoding string inputs in model inputs (see below) and instantiating/saving python and | |
| "Fast" tokenizers either from a local file or directory or from a pretrained tokenizer provided by the library | |
| (downloaded from HuggingFace's AWS S3 repository). |