chunked/nltk_chunking/_deepspeed/chunk_159.txt

Set the allgather_bucket_size and reduce_bucket_size values to 2e8 in the ZeRO-2 configuration file to get better performance on a single GPU.