| deepspeed --num_gpus=2 examples/pytorch/translation/run_translation.py \ | |
| --deepspeed tests/deepspeed/ds_config_zero3.json \ | |
| --model_name_or_path google-t5/t5-small --per_device_train_batch_size 1 \ | |
| --output_dir output_dir --overwrite_output_dir --fp16 \ | |
| --do_train --max_train_samples 500 --num_train_epochs 1 \ | |
| --dataset_name wmt16 --dataset_config "ro-en" \ | |
| --source_lang en --target_lang ro | |
| To deploy DeepSpeed on a single GPU, add the --num_gpus parameter. |