chunked/nltk_chunking/_deepspeed/chunk_84.txt

Another consideration for ZeRO-3 is if you have multiple GPUs, no single GPU has all the parameters unless it's the parameters for the currently executing layer.