Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
raw
history blame contribute delete
122 Bytes
[SEP]'
As you can see, the tokenizer added two special tokens - CLS and SEP (classifier and separator) - to the sentence.