| Usage example | |
| thon | |
| import torch | |
| from transformers import AutoModel, AutoTokenizer | |
| bertweet = AutoModel.from_pretrained("vinai/bertweet-base") | |
| For transformers v4.x+: | |
| tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", use_fast=False) | |
| For transformers v3.x: | |
| tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base") | |
| INPUT TWEET IS ALREADY NORMALIZED! |