jusjinuk
/

Meta-Llama-3-8B-2bit-SqueezeLLM

Model card Files Files and versions Community

Model Card

Base model: meta-llama/Meta-Llama-3-8B
Quantization method: SqueezeLLM
Target bit-width: 2
Backend kernel: Any-Precision-LLM kernel (ap-gemv)
Calibration data: RedPajama (1024 sentences / 4096 tokens)
Calibration objective: Next-token prediction

How to run

Follow the instruction in https://github.com/snu-mllab/GuidedQuant.

References

Model Paper

Downloads last month: 3

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for jusjinuk/Meta-Llama-3-8B-2bit-SqueezeLLM

Base model

meta-llama/Meta-Llama-3-8B

Quantized

(262)

this model

Collection including jusjinuk/Meta-Llama-3-8B-2bit-SqueezeLLM

Pre-trained models (SqueezeLLM)

15 items • Updated Jun 19