Update README.md
Browse files
README.md
CHANGED
@@ -11,42 +11,137 @@ license_link: https://huggingface.co/MTSAIR/Kodify-Nano-GGUF/blob/main/Apache%20
|
|
11 |
|
12 |
# Kodify-Nano-GGUF 🤖
|
13 |
|
14 |
-
Kodify-Nano-GGUF -
|
15 |
|
16 |
-
Kodify-Nano-GGUF -
|
17 |
|
18 |
-
## Download Models
|
19 |
|
20 |
-
|
21 |
-
|
22 |
-
- Kodify_Nano_q8_0.gguf (high quality)
|
23 |
-
- Kodify_Nano.gguf (best quality, unquantized)
|
24 |
|
25 |
-
|
|
|
|
|
|
|
|
|
|
|
26 |
|
27 |
```bash
|
28 |
-
|
29 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
```
|
31 |
|
32 |
-
|
|
|
|
|
|
|
|
|
|
|
33 |
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
https://ollama.com/download
|
36 |
|
37 |
-
2.
|
38 |
|
|
|
|
|
39 |
```
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
```
|
46 |
|
47 |
-
3. Create and run model:
|
48 |
-
ollama create kodify-nano -f Modelfile
|
49 |
-
ollama run kodify-nano "Write a Python function to check prime numbers"
|
50 |
|
51 |
## Python Integration
|
52 |
|
@@ -76,8 +171,6 @@ print(response['response'])
|
|
76 |
|
77 |
## Usage Examples
|
78 |
|
79 |
-
### Code Generation
|
80 |
-
|
81 |
```python
|
82 |
response = ollama.generate(
|
83 |
model="kodify-nano",
|
|
|
11 |
|
12 |
# Kodify-Nano-GGUF 🤖
|
13 |
|
14 |
+
Kodify-Nano-GGUF - GGUF версия модели [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), оптимизированная для CPU/GPU-инференса и использованием Ollama/llama.cpp. Легковесная LLM для задач разработки кода с минимальными ресурсами.
|
15 |
|
16 |
+
Kodify-Nano-GGUF - GGUF version of [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), optimized for CPU/GPU inference with Ollama/llama.cpp. Lightweight LLM for code development tasks with minimal resource requirements.
|
17 |
|
|
|
18 |
|
19 |
+
## Using the Image
|
20 |
+
You can run Kodify Nano on OLLAMA in two ways:
|
|
|
|
|
21 |
|
22 |
+
1. **Using Docker**
|
23 |
+
2. **Locally** (provides faster responses than Docker)
|
24 |
+
|
25 |
+
### Method 1: Running Kodify Nano on OLLAMA in Docker
|
26 |
+
|
27 |
+
#### Without NVIDIA GPU:
|
28 |
|
29 |
```bash
|
30 |
+
docker run -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama
|
31 |
+
```
|
32 |
+
|
33 |
+
#### With NVIDIA GPU:
|
34 |
+
|
35 |
+
```bash
|
36 |
+
docker run --runtime nvidia -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama
|
37 |
+
```
|
38 |
+
|
39 |
+
> **Important:**
|
40 |
+
> - Ensure Docker is installed and running
|
41 |
+
> - If port 8985 is occupied, replace it with any available port and update plugin configuration
|
42 |
+
|
43 |
+
#### Load the model:
|
44 |
+
|
45 |
+
```bash
|
46 |
+
docker exec ollama ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF
|
47 |
```
|
48 |
|
49 |
+
#### Rename the model:
|
50 |
+
```bash
|
51 |
+
docker exec ollama ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano
|
52 |
+
```
|
53 |
+
|
54 |
+
#### Start the model:
|
55 |
|
56 |
+
```bash
|
57 |
+
docker exec ollama ollama run kodify_nano
|
58 |
+
```
|
59 |
+
---
|
60 |
+
|
61 |
+
### Method 2: Local Kodify Nano on OLLAMA
|
62 |
+
|
63 |
+
1. **Download OLLAMA:**
|
64 |
https://ollama.com/download
|
65 |
|
66 |
+
2. **Set the port:**
|
67 |
|
68 |
+
```bash
|
69 |
+
export OLLAMA_HOST=0.0.0.0:8985
|
70 |
```
|
71 |
+
|
72 |
+
> **Note:** If port 8985 is occupied, replace it and update plugin configuration
|
73 |
+
|
74 |
+
3. **Start OLLAMA server:**
|
75 |
+
|
76 |
+
```bash
|
77 |
+
ollama serve &
|
78 |
+
```
|
79 |
+
|
80 |
+
4. **Download the model:**
|
81 |
+
|
82 |
+
```bash
|
83 |
+
ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF
|
84 |
+
```
|
85 |
+
|
86 |
+
5. **Rename the model:**
|
87 |
+
|
88 |
+
```bash
|
89 |
+
ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano
|
90 |
+
```
|
91 |
+
|
92 |
+
6. **Run the model:**
|
93 |
+
|
94 |
+
```bash
|
95 |
+
ollama run kodify_nano
|
96 |
+
```
|
97 |
+
|
98 |
+
## Plugin Installation
|
99 |
+
|
100 |
+
### For Visual Studio Code
|
101 |
+
|
102 |
+
1. Download the [latest Kodify plugin](https://mts.ai/ru/product/kodify/?utm_source=huggingface&utm_medium=pr&utm_campaign=post#models) for VS Code.
|
103 |
+
2. Open the **Extensions** panel on the left sidebar.
|
104 |
+
3. Click **Install from VSIX...** and select the downloaded plugin file.
|
105 |
+
|
106 |
+
### For JetBrains IDEs
|
107 |
+
|
108 |
+
1. Download the [latest Kodify plugin](https://mts.ai/ru/product/kodify/?utm_source=huggingface&utm_medium=pr&utm_campaign=post#models) for JetBrains.
|
109 |
+
2. Open the IDE and go to **Settings > Plugins**.
|
110 |
+
3. Click the gear icon (⚙️) and select **Install Plugin from Disk...**.
|
111 |
+
4. Choose the downloaded plugin file.
|
112 |
+
5. Restart the IDE when prompted.
|
113 |
+
|
114 |
+
---
|
115 |
+
|
116 |
+
### Changing the Port in Plugin Settings (for Visual Studio Code and JetBrains)
|
117 |
+
|
118 |
+
If you changed the Docker port from `8985`, update the plugin's `config.json`:
|
119 |
+
|
120 |
+
1. Open any file in the IDE.
|
121 |
+
2. Open the Kodify sidebar:
|
122 |
+
- **VS Code**: `Ctrl+L` (`Cmd+L` on Mac).
|
123 |
+
- **JetBrains**: `Ctrl+J` (`Cmd+J` on Mac).
|
124 |
+
3. Access the `config.json` file:
|
125 |
+
- **Method 1**: Click **Open Settings** (VS Code) or **Kodify Config** (JetBrains), then navigate to **Configuration > Chat Settings > Open Config File**.
|
126 |
+
- **Method 2**: Click the gear icon (⚙️) in the Kodify sidebar.
|
127 |
+
4. Modify the `apiBase` port under `tabAutocompleteModel` and `models`.
|
128 |
+
5. Save the file (`Ctrl+S` or **File > Save**).
|
129 |
+
|
130 |
+
---
|
131 |
+
|
132 |
+
|
133 |
+
## Available quantization variants:
|
134 |
+
- Kodify_Nano_q4_k_s.gguf (balanced)
|
135 |
+
- Kodify_Nano_q8_0.gguf (high quality)
|
136 |
+
- Kodify_Nano.gguf (best quality, unquantized)
|
137 |
+
|
138 |
+
Download using huggingface_hub:
|
139 |
+
|
140 |
+
```bash
|
141 |
+
pip install huggingface-hub
|
142 |
+
python -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='MTSAIR/Kodify-Nano-GGUF', filename='Kodify_Nano_q4_k_s.gguf', local_dir='./models')"
|
143 |
```
|
144 |
|
|
|
|
|
|
|
145 |
|
146 |
## Python Integration
|
147 |
|
|
|
171 |
|
172 |
## Usage Examples
|
173 |
|
|
|
|
|
174 |
```python
|
175 |
response = ollama.generate(
|
176 |
model="kodify-nano",
|