py | |
quantized_model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", quantization_config=gptq_config) | |
If you're running out of memory because a dataset is too large, disk offloading is not supported. |
py | |
quantized_model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", quantization_config=gptq_config) | |
If you're running out of memory because a dataset is too large, disk offloading is not supported. |