how to use the gguf models instead of default model which is starts downloading when we use command - ggc h2 ?
#1
by
hakaishin008
- opened
I did installed all required packages with their repo but when i used the command which you mentioned it started downloading the original model without giving me any option to select any gguf model, so can you please guide me on this calcius
that's not the original model; which is f16, not bf16; make sure you could run it first; those gguf files work if you dequant it or convert it back to safetensors or pt/pth; at this stage, the gguf in this repo is for developers whom are willing to work on the engine; just be patient and wait