Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
deepspeed --num_gpus 8 --num_nodes 2 --hostfile hostfile --master_addr hostname1 --master_port=9901 \
your_program.py <normal cl args> --deepspeed ds_config.json
Check out the Resource Configuration (multi-node) guide for more details about configuring multi-node compute resources.