Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
The format is: Salesforce/codegen-{size}-{data}, where
size: 350M, 2B, 6B, 16B
data:
nl: Pre-trained on the Pile
multi: Initialized with nl, then further pre-trained on multiple programming languages data
mono: Initialized with multi, then further pre-trained on Python data
For example, Salesforce/codegen-350M-mono offers a 350 million-parameter checkpoint pre-trained sequentially on the Pile, multiple programming languages, and Python.