| You'll also need to specify the position of the` token: | |
| from transformers import AutoTokenizer | |
| tokenizer = AutoTokenizer.from_pretrained("username/my_awesome_eli5_mlm_model") | |
| inputs = tokenizer(text, return_tensors="tf") | |
| mask_token_index = tf.where(inputs["input_ids"] == tokenizer.mask_token_id)[0, 1] | |
| Pass your inputs to the model and return the logits of the masked token: | |
| from transformers import TFAutoModelForMaskedLM | |
| model = TFAutoModelForMaskedLM.from_pretrained("username/my_awesome_eli5_mlm_model") | |
| logits = model(**inputs).logits | |
| mask_token_logits = logits[0, mask_token_index, :] | |
| Then return the three masked tokens with the highest probability and print them out: | |
| top_3_tokens = tf.math.top_k(mask_token_logits, 3).indices.numpy() | |
| for token in top_3_tokens: | |
| print(text.replace(tokenizer.mask_token, tokenizer.decode([token]))) | |
| The Milky Way is a spiral galaxy. |