InfoTokenizers
/

tokenizers

Model card Files Files and versions Community

tokenizers / bytelevel

Ctrl+K

Ctrl+K

3 contributors

History: 1 commit

pietrolesci's picture

Upload folder using huggingface_hub

592b2b4 verified about 1 month ago

special_tokens_map.json

65 Bytes

Upload folder using huggingface_hub about 1 month ago
tokenizer.json

5.42 kB

Upload folder using huggingface_hub about 1 month ago
tokenizer_config.json

685 Bytes

Upload folder using huggingface_hub about 1 month ago