Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
thon
inputs = tokenizer(sequence)
The tokenizer returns a dictionary with all the arguments necessary for its corresponding model to work properly.