yaml
{
   "zero_force_ds_cpu_optimizer": false
}

DeepSpeed supports the LRRangeTest, OneCycle, WarmupLR and WarmupDecayLR learning rate schedulers.