chunked/nltk_chunking/main_classes_tokenizer/chunk_7.txt

[PreTrainedTokenizer] and [PreTrainedTokenizerFast] thus implement the main
methods for using all the tokenizers:

Tokenizing (splitting strings in sub-word token strings), converting tokens strings to ids and back, and
  encoding/decoding (i.e., tokenizing and converting to integers).