The quantization_config wasn't added in the json hence vllm hasn't able to run this !
LGTM! Thanks
· Sign up or log in to comment