Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
To ensure that the tokenizer works correctly, it is recommended to first create a script in the original repository
that inputs a string and returns the `input_ids``.