File size: 3,971 Bytes
dcdde00 c071d57 dcdde00 7a10142 73af091 2bcf7cd 73af091 c071d57 dcdde00 c651941 c071d57 dcdde00 c651941 dcdde00 c071d57 dcdde00 c071d57 dcdde00 c071d57 dcdde00 c071d57 dcdde00 c071d57 dcdde00 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 |
---
base_model:
- huihui-ai/Arcee-Blitz-abliterated
- darkc0de/BlackXorDolphTronGOAT
- PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
- aixonlab/Eurydice-24b-v3.5
- Gryphe/Pantheon-RP-1.8-24b-Small-3.1
- TheDrummer/Cydonia-24B-v2.1
- ReadyArt/Broken-Tutu-24B
- ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
library_name: transformers
tags:
- mergekit
- merge
---
# Erotophobia-24-v2.0
My second merge! Yayay!
*Technically my third but we'll ignore the first failed model :(*
This model is just headache to do, it really is! It wasted me $20 trying to fit darkc0de/BlackXorDolphTronGOAT because it's a `float32` dtype!
I didn't know I can downcast the model into `dfloat16` first until at the very end there.
But, this model has the downcast in the [df16](df16) directory for your pleasure.
> Still testing and awaiting GGUF. Please kindly give your patience. Thank you <3
## Philosophy
### Mind
Fusion of darkc0de's fusions merge, and Acree's DeepSeek distill abliterated.
I think this is a good base. The Xortron is the top performing at UGI leaderboard and Arcee has the DeepSeek distill and updated world information.
### Heart
Karcher with all 2503 base, using updated Dan's and Eurydice, and reintroduce Pantheon.
I feel like this will improve the roleplay. All has the models have unique characteristics of their own, and hopefully the Karcher mean can find a nice center.
### Soul
Stock on Sleep's Omega Gaslight and Broken Tutu, since both based on Cydonia 2.1 and have the BlackSheep model in their merge.
I'm horny...
## Merge Details
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
### Merge Method
This model was merged using multi-stage model merging method:
- **Mind** was merged using the [Arcee Fusion](https://www.arcee.ai/blog/meet-mergekit-v0-1-arcee-fusion-expanded-model-support-multi-gpu-acceleration) merge method using huihui-ai/Arcee-Blitz-abliterated as a base.
- **Heart** was merged using the [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) merge method.
- **Soul** was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using TheDrummer/Cydonia-24B-v2.1 as a base.
Fially, this model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using **Mind** as a base.
The partial merge results (**Mind**, **Heart**, and **Soul**) are available inside the [intermediates](intermediates) directory.
### Models Merged
The following models were included in the merge:
* **Mind**
- darkc0de/BlackXorDolphTronGOAT
* **Soul**
- PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
- aixonlab/Eurydice-24b-v3.5
- Gryphe/Pantheon-RP-1.8-24b-Small-3.1
* **Heart**
- ReadyArt/Broken-Tutu-24B
- ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
### Configuration
The following YAML configuration was used to produce this model (using `mergekit-multi`):
```yaml
name: Mind
merge_method: arcee_fusion
dtype: bfloat16
tokenizer:
source: union
chat_template: auto
base_model: huihui-ai/Arcee-Blitz-abliterated
models:
- model: darkc0de/BlackXorDolphTronGOAT
---
name: Heart
merge_method: karcher
tokenizer:
source: union
chat_template: auto
parameters:
max_iter: 1000
models:
- model: PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
- model: aixonlab/Eurydice-24b-v3.5
- model: Gryphe/Pantheon-RP-1.8-24b-Small-3.1
---
name: Soul
merge_method: model_stock
tokenizer:
source: union
chat_template: auto
base_model: TheDrummer/Cydonia-24B-v2.1
models:
- model: ReadyArt/Broken-Tutu-24B
- model: ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
---
merge_method: della
tokenizer:
source: union
chat_template: auto
base_model: Mind
models:
- model: Mind
- model: Heart
parameters:
weight: 0.6
- model: Soul
parameters:
weight: 0.4
parameters:
density: 0.7
epsilon: 0.2
lamda: 1.1
```
|