Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
If you have NVMe and ZeRO-3 setup, experiment with offloading to the NVMe (estimate the memory requirements for your model).