Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,48 @@
|
|
1 |
-
---
|
2 |
-
license:
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
base_model:
|
4 |
+
- arcee-ai/Homunculus
|
5 |
+
pipeline_tag: text-generation
|
6 |
+
tags:
|
7 |
+
- OpenVINO
|
8 |
+
- Optimum-Intel
|
9 |
+
- OpenArc
|
10 |
+
---
|
11 |
+
|
12 |
+
My Project [OpenArc](https://github.com/SearchSavior/OpenArc), an inference engine for OpenVINO, now supports this model and serves inference over OpenAI compatible endpoints for text to text *and* text with vision!
|
13 |
+
|
14 |
+
We have a growing Discord community of others interested in using Intel for AI/ML.
|
15 |
+
|
16 |
+
[](https://discord.gg/maMY7QjG)
|
17 |
+
|
18 |
+
- Find documentation on the Optimum-CLI export process [here](https://huggingface.co/docs/optimum/main/en/intel/openvino/export)
|
19 |
+
- Use my HF space [Echo9Zulu/Optimum-CLI-Tool_tool](https://huggingface.co/spaces/Echo9Zulu/Optimum-CLI-Tool_tool) to build commands and execute locally
|
20 |
+
|
21 |
+
---
|
22 |
+
|
23 |
+
## This repo contains OpenVINO quantizied versions of arcee-ai/Homunculus.
|
24 |
+
|
25 |
+
I reccomend starting with **Homunculus-int4_asym-awq-se-ov**
|
26 |
+
|
27 |
+
To download individual models from this repo use the provided snippet:
|
28 |
+
|
29 |
+
```
|
30 |
+
from huggingface_hub import snapshot_download
|
31 |
+
|
32 |
+
repo_id = "Echo9Zulu/Homunculus-OpenVINO"
|
33 |
+
|
34 |
+
# Choose the weights you want
|
35 |
+
repo_directory = "Homunculus-int4_asym-awq-se-ov"
|
36 |
+
|
37 |
+
# Where you want to save it
|
38 |
+
local_dir = "./Echo9Zulu_Homunculus/Homunculus-int4_asym-awq-se-ov"
|
39 |
+
|
40 |
+
snapshot_download(
|
41 |
+
repo_id=repo_id,
|
42 |
+
allow_patterns=[f"{repo_directory}/*"],
|
43 |
+
local_dir=local_dir,
|
44 |
+
local_dir_use_symlinks=True
|
45 |
+
)
|
46 |
+
|
47 |
+
print("Download complete!")
|
48 |
+
```
|