---
language:
- en
- fr
- de
- es
- it
- pt
license: apache-2.0
library_name: transformers
inference: false
extra_gated_description: >-
  If you want to learn more about how we process your personal data, please read
  our <a href="https://www.almawave.com/privacy-policy/">Privacy Policy</a>.
tags:
- vllm
base_model:
- Almawave/Velvet-14B
---

## DESCRIPTION
**This model does not represent the intended quality of the original product.**

To perform this quantization, we started with llama.cpp as our base, modifying the file convert_hf_to_gguf_update.py to support this model.
For modifying this file, we based our work on what was seen in the PR https://github.com/ggerganov/llama.cpp/pull/11716.

**Note:** As of today, llama.cpp does not support this model or this chat template https://github.com/ggerganov/llama.cpp/pull/11716. 

## PROMPT FORMAT

Basic prompt format:

```
<s><instruction>{prompt}</instruction>
```

Prompt format with system message:

```
<s><instruction>{system_prompt}
{prompt}</instruction>
```

## DOWNLOAD

| Quant  | Link |
| ------ | ---- |
| BF16   | [Velvet-14B-BF16](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-BF16.gguf) |
| F16    | [Velvet-14B-F16.gguf](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-F16.gguf) |
| Q4_K_M | [Velvet-14B-Q4_K_M](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-Q4_K_M.gguf) |
| Q4_K_S | [Velvet-14B-Q4_K_S](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-Q4_K_S.gguf) |
| Q5_K_M | [Velvet-14B-Q5_K_M](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-Q5_K_M.gguf) |
| Q6_K   | [Velvet-14B-Q6_K.gguf](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-Q6_K.gguf) |
| Q8_0   | [Velvet-14B-Q8_0.gguf](https://huggingface.co/SistInf/Velvet-14B-GGUF/blob/main/Velvet-14B-Q8_0.gguf) |


Original Model: https://huggingface.co/Almawave/Velvet-14B

### License
Velvet-14B and Velvet-2B are made available under the Apache 2.0 license