Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
For example, consider these two sequences:
thon
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained("google-bert/bert-base-cased")
sequence_a = "This is a short sequence."