mlx-community

/

gemma-3-12b-it-qat-4bit

like 5

Follow

MLX Community 4.77k

Converted with this command:
python -m mlx_vlm.convert --hf-path /Volumes/T7/Models/hf-models/gemma-3-12b-it-qat-q4_0-unquantized --mlx-path gemma-3-12b-it-qat-4bit --dtype bfloat16 -q --q-bits 4

Note that the dtype is bfloat16. This model was tested, and confirmed working with mlx_vlm 0.1.25 CLI, and LM Studio. Note the config.json change, the conversion process made this change, and I saw that it was necessary to keep this change to load the model with mlx_vlm