For large models and multiple GPUs, this is an expensive in terms of memory and speed.