Polushinm commited on
Commit
4c5690c
·
verified ·
1 Parent(s): 494a20a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +116 -23
README.md CHANGED
@@ -11,42 +11,137 @@ license_link: https://huggingface.co/MTSAIR/Kodify-Nano-GGUF/blob/main/Apache%20
11
 
12
  # Kodify-Nano-GGUF 🤖
13
 
14
- Kodify-Nano-GGUF - 4-битная квантизированная GGUF версия модели [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), оптимизированная для CPU-инференса. Легковесная LLM для задач разработки кода с минимальными ресурсами.
15
 
16
- Kodify-Nano-GGUF - 4-bit quantized GGUF version of [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), optimized for CPU inference. Lightweight LLM for code development tasks with minimal resource requirements.
17
 
18
- ## Download Models
19
 
20
- Available quantization variants:
21
- - Kodify_Nano_q4_k_s.gguf (balanced)
22
- - Kodify_Nano_q8_0.gguf (high quality)
23
- - Kodify_Nano.gguf (best quality, unquantized)
24
 
25
- Download using huggingface_hub:
 
 
 
 
 
26
 
27
  ```bash
28
- pip install huggingface-hub
29
- python -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='MTSAIR/Kodify-Nano-GGUF', filename='Kodify_Nano_q4_k_s.gguf', local_dir='./models')"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  ```
31
 
32
- ## Using with Ollama
 
 
 
 
 
33
 
34
- 1. Install Ollama:
 
 
 
 
 
 
 
35
  https://ollama.com/download
36
 
37
- 2. Create Modelfile:
38
 
 
 
39
  ```
40
- FROM ./models/Kodify_Nano_q4_k_s.gguf
41
- PARAMETER temperature 0.4
42
- PARAMETER top_p 0.8
43
- PARAMETER num_ctx 8192
44
- TEMPLATE """<s>[INST] {{ .System }} {{ .Prompt }} [/INST]"""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ```
46
 
47
- 3. Create and run model:
48
- ollama create kodify-nano -f Modelfile
49
- ollama run kodify-nano "Write a Python function to check prime numbers"
50
 
51
  ## Python Integration
52
 
@@ -76,8 +171,6 @@ print(response['response'])
76
 
77
  ## Usage Examples
78
 
79
- ### Code Generation
80
-
81
  ```python
82
  response = ollama.generate(
83
  model="kodify-nano",
 
11
 
12
  # Kodify-Nano-GGUF 🤖
13
 
14
+ Kodify-Nano-GGUF - GGUF версия модели [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), оптимизированная для CPU/GPU-инференса и использованием Ollama/llama.cpp. Легковесная LLM для задач разработки кода с минимальными ресурсами.
15
 
16
+ Kodify-Nano-GGUF - GGUF version of [MTSAIR/Kodify-Nano](https://huggingface.co/MTSAIR/Kodify-Nano), optimized for CPU/GPU inference with Ollama/llama.cpp. Lightweight LLM for code development tasks with minimal resource requirements.
17
 
 
18
 
19
+ ## Using the Image
20
+ You can run Kodify Nano on OLLAMA in two ways:
 
 
21
 
22
+ 1. **Using Docker**
23
+ 2. **Locally** (provides faster responses than Docker)
24
+
25
+ ### Method 1: Running Kodify Nano on OLLAMA in Docker
26
+
27
+ #### Without NVIDIA GPU:
28
 
29
  ```bash
30
+ docker run -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama
31
+ ```
32
+
33
+ #### With NVIDIA GPU:
34
+
35
+ ```bash
36
+ docker run --runtime nvidia -e OLLAMA_HOST=0.0.0.0:8985 -p 8985:8985 --name ollama -d ollama/ollama
37
+ ```
38
+
39
+ > **Important:**
40
+ > - Ensure Docker is installed and running
41
+ > - If port 8985 is occupied, replace it with any available port and update plugin configuration
42
+
43
+ #### Load the model:
44
+
45
+ ```bash
46
+ docker exec ollama ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF
47
  ```
48
 
49
+ #### Rename the model:
50
+ ```bash
51
+ docker exec ollama ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano
52
+ ```
53
+
54
+ #### Start the model:
55
 
56
+ ```bash
57
+ docker exec ollama ollama run kodify_nano
58
+ ```
59
+ ---
60
+
61
+ ### Method 2: Local Kodify Nano on OLLAMA
62
+
63
+ 1. **Download OLLAMA:**
64
  https://ollama.com/download
65
 
66
+ 2. **Set the port:**
67
 
68
+ ```bash
69
+ export OLLAMA_HOST=0.0.0.0:8985
70
  ```
71
+
72
+ > **Note:** If port 8985 is occupied, replace it and update plugin configuration
73
+
74
+ 3. **Start OLLAMA server:**
75
+
76
+ ```bash
77
+ ollama serve &
78
+ ```
79
+
80
+ 4. **Download the model:**
81
+
82
+ ```bash
83
+ ollama pull hf.co/MTSAIR/Kodify-Nano-GGUF
84
+ ```
85
+
86
+ 5. **Rename the model:**
87
+
88
+ ```bash
89
+ ollama cp hf.co/MTSAIR/Kodify-Nano-GGUF kodify_nano
90
+ ```
91
+
92
+ 6. **Run the model:**
93
+
94
+ ```bash
95
+ ollama run kodify_nano
96
+ ```
97
+
98
+ ## Plugin Installation
99
+
100
+ ### For Visual Studio Code
101
+
102
+ 1. Download the [latest Kodify plugin](https://mts.ai/ru/product/kodify/?utm_source=huggingface&utm_medium=pr&utm_campaign=post#models) for VS Code.
103
+ 2. Open the **Extensions** panel on the left sidebar.
104
+ 3. Click **Install from VSIX...** and select the downloaded plugin file.
105
+
106
+ ### For JetBrains IDEs
107
+
108
+ 1. Download the [latest Kodify plugin](https://mts.ai/ru/product/kodify/?utm_source=huggingface&utm_medium=pr&utm_campaign=post#models) for JetBrains.
109
+ 2. Open the IDE and go to **Settings > Plugins**.
110
+ 3. Click the gear icon (⚙️) and select **Install Plugin from Disk...**.
111
+ 4. Choose the downloaded plugin file.
112
+ 5. Restart the IDE when prompted.
113
+
114
+ ---
115
+
116
+ ### Changing the Port in Plugin Settings (for Visual Studio Code and JetBrains)
117
+
118
+ If you changed the Docker port from `8985`, update the plugin's `config.json`:
119
+
120
+ 1. Open any file in the IDE.
121
+ 2. Open the Kodify sidebar:
122
+ - **VS Code**: `Ctrl+L` (`Cmd+L` on Mac).
123
+ - **JetBrains**: `Ctrl+J` (`Cmd+J` on Mac).
124
+ 3. Access the `config.json` file:
125
+ - **Method 1**: Click **Open Settings** (VS Code) or **Kodify Config** (JetBrains), then navigate to **Configuration > Chat Settings > Open Config File**.
126
+ - **Method 2**: Click the gear icon (⚙️) in the Kodify sidebar.
127
+ 4. Modify the `apiBase` port under `tabAutocompleteModel` and `models`.
128
+ 5. Save the file (`Ctrl+S` or **File > Save**).
129
+
130
+ ---
131
+
132
+
133
+ ## Available quantization variants:
134
+ - Kodify_Nano_q4_k_s.gguf (balanced)
135
+ - Kodify_Nano_q8_0.gguf (high quality)
136
+ - Kodify_Nano.gguf (best quality, unquantized)
137
+
138
+ Download using huggingface_hub:
139
+
140
+ ```bash
141
+ pip install huggingface-hub
142
+ python -c "from huggingface_hub import hf_hub_download; hf_hub_download(repo_id='MTSAIR/Kodify-Nano-GGUF', filename='Kodify_Nano_q4_k_s.gguf', local_dir='./models')"
143
  ```
144
 
 
 
 
145
 
146
  ## Python Integration
147
 
 
171
 
172
  ## Usage Examples
173
 
 
 
174
  ```python
175
  response = ollama.generate(
176
  model="kodify-nano",