| deepspeed --num_gpus 8 --num_nodes 2 --hostfile hostfile --master_addr hostname1 --master_port=9901 \ | |
| your_program.py <normal cl args> --deepspeed ds_config.json | |
| Check out the Resource Configuration (multi-node) guide for more details about configuring multi-node compute resources. |