rdsm/openhands-lm-32b-v0.1-mlx-mixed-3_6bit
MLX Mixed Quantization 3/6bits, using --quant-predicate mixed_3_6 ~4bpw
The Model rdsm/openhands-lm-32b-v0.1-mlx-mixed-3_6bit was converted to MLX format from all-hands/openhands-lm-32b-v0.1 using mlx-lm version 0.22.2.
Note: Qwen 2.5 0.5b / 1.5b Instruct seems to work fine as draft models.
- Downloads last month
- 7
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The HF Inference API does not support text-generation models for mlx
library.
Model tree for rdsm/openhands-lm-32b-v0.1-mlx-mixed-3_6bit
Base model
Qwen/Qwen2.5-32B
Finetuned
Qwen/Qwen2.5-Coder-32B
Finetuned
Qwen/Qwen2.5-Coder-32B-Instruct
Finetuned
all-hands/openhands-lm-32b-v0.1