deepspeed --num_gpus=2 examples/pytorch/translation/run_translation.py \ | |
--deepspeed tests/deepspeed/ds_config_zero3.json \ | |
--model_name_or_path google-t5/t5-small --per_device_train_batch_size 1 \ | |
--output_dir output_dir --overwrite_output_dir --fp16 \ | |
--do_train --max_train_samples 500 --num_train_epochs 1 \ | |
--dataset_name wmt16 --dataset_config "ro-en" \ | |
--source_lang en --target_lang ro | |
To deploy DeepSpeed on a single GPU, add the --num_gpus parameter. |