File size: 370 Bytes
5fa1a76 |
1 2 3 4 5 6 7 8 9 10 11 |
Usage example
thon
import torch
from transformers import AutoModel, AutoTokenizer
bertweet = AutoModel.from_pretrained("vinai/bertweet-base")
For transformers v4.x+:
tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", use_fast=False)
For transformers v3.x:
tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base")
INPUT TWEET IS ALREADY NORMALIZED! |