Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files Community
Zeb commited on
Commit
84b51c9
·
1 Parent(s): dcaf536

Fix tokenizer names

Browse files
Files changed (12) hide show
  1. {ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/special_tokens_map.json +0 -0
  2. {ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/tokenizer.json +0 -0
  3. {ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/tokenizer_config.json +0 -0
  4. {ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/vocab.json +0 -0
  5. {ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/special_tokens_map.json +0 -0
  6. {ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/tokenizer.json +0 -0
  7. {ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/tokenizer_config.json +0 -0
  8. {ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/vocab.json +0 -0
  9. {ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/special_tokens_map.json +0 -0
  10. {ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/tokenizer.json +0 -0
  11. {ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/tokenizer_config.json +0 -0
  12. {ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/vocab.json +0 -0
{ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/special_tokens_map.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/tokenizer.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/tokenizer_config.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_16000 → MultiByteSpanEntropyCombinedSeeding_16000}/vocab.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/special_tokens_map.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/tokenizer.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/tokenizer_config.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_32000 → MultiByteSpanEntropyCombinedSeeding_32000}/vocab.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/special_tokens_map.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/tokenizer.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/tokenizer_config.json RENAMED
File without changes
{ByteSpanEntropyCombinedSeedingmulti_64000 → MultiByteSpanEntropyCombinedSeeding_64000}/vocab.json RENAMED
File without changes