Let's look at the different precisions:

float32: PyTorch convention on model initialization is to load models in float32, no matter with which dtype the model weights were stored.