allgather_bucket_size and reduce_bucket_size trade off available GPU memory for communication speed.