Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,7 @@ recipe = [
|
|
24 |
### vllm
|
25 |
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e.,
|
26 |
```
|
27 |
-
vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
28 |
```
|
29 |
# Qwen3-Coder-30B-A3B-Instruct
|
30 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|
|
|
24 |
### vllm
|
25 |
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e.,
|
26 |
```
|
27 |
+
vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-4bit --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
28 |
```
|
29 |
# Qwen3-Coder-30B-A3B-Instruct
|
30 |
<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
|