Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
If we use tokenize=True, which is the default setting,
that string will also be tokenized for us.