Updated!
Please grab "v2" quants remade with the new tokenizer settings to fix the endless generation issues.

SillyTavern
The complete AIO recommended preset:
v2-SillyTavern-Presets-AIO-2024-12-28.json

⛶ [Expand/hide] Example setup.

Example setup in SillyTavern...

GGUF

Model size

12.2B params

Architecture

llama

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Lewdiculous/Captain-Eris_Twighlight-V0.420-12B-GGUF-ARM-Imatrix

Base model

Quantized

(10)

this model

Collection including Lewdiculous/Captain-Eris_Twighlight-V0.420-12B-GGUF-ARM-Imatrix