File size: 766 Bytes
5fa1a76 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
Usage example
thon
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "Salesforce/codegen-350M-mono"
model = AutoModelForCausalLM.from_pretrained(checkpoint)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
text = "def hello_world():"
completion = model.generate(**tokenizer(text, return_tensors="pt"))
print(tokenizer.decode(completion[0]))
def hello_world():
print("Hello World")
hello_world()
Resources
Causal language modeling task guide
CodeGenConfig
[[autodoc]] CodeGenConfig
- all
CodeGenTokenizer
[[autodoc]] CodeGenTokenizer
- save_vocabulary
CodeGenTokenizerFast
[[autodoc]] CodeGenTokenizerFast
CodeGenModel
[[autodoc]] CodeGenModel
- forward
CodeGenForCausalLM
[[autodoc]] CodeGenForCausalLM
- forward |