Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
If we decode the previous sequence of ids,
thon
decoded_sequence = tokenizer.decode(encoded_sequence)
we will see
thon
print(decoded_sequence)
[CLS] A Titan RTX has 24GB of VRAM [SEP]
because this is the way a [BertModel] is going to expect its inputs.