If you loaded your model on the CPU, make sure to move it to a GPU device first.