metadata
library_name: vllm
base_model:
- moonshotai/Kimi-K2-Instruct
pipeline_tag: text-generation
tags:
- neuralmagic
- redhat
- llmcompressor
- quantized
- INT4
- GPTQ
Preliminary version of the model
Evaluations
- GSM8k, 5-shot via lm-evaluation-harness
moonshotai/Kimi-K2-Instruct = 94.92
RedHatAI/Kimi-K2-Instruct-quantized.w4a16 (this model) = 94.84
More evals coming very soon...