| Here's an example: | |
| thon | |
| from transformers import GPT2Tokenizer, GPT2LMHeadModel | |
| tokenizer = GPT2Tokenizer.from_pretrained("openai-community/gpt2") | |
| model = GPT2LMHeadModel.from_pretrained("openai-community/gpt2") | |
| inputs = tokenizer("Hello, my dog is cute and ", return_tensors="pt") | |
| generation_output = model.generate(**inputs, return_dict_in_generate=True, output_scores=True) | |
| The generation_output object is a [~generation.GenerateDecoderOnlyOutput], as we can | |
| see in the documentation of that class below, it means it has the following attributes: | |
| sequences: the generated sequences of tokens | |
| scores (optional): the prediction scores of the language modelling head, for each generation step | |
| hidden_states (optional): the hidden states of the model, for each generation step | |
| attentions (optional): the attention weights of the model, for each generation step | |
| Here we have the scores since we passed along output_scores=True, but we don't have hidden_states and | |
| attentions because we didn't pass output_hidden_states=True or output_attentions=True. |