10 6 18

Emin Temiz PRO

etemiz

https://pickabrain.ai

AI & ML interests

Alignment

Recent Activity

new activity about 11 hours ago

mradermacher/Llama-4-Maverick-17B-128E-Instruct-GGUF:bug free

posted an update 5 days ago

Grok 3 Human Alignment Score: 42 It is better in health, nutrition, fasting compared to Grok 2. About the same in liberating tech like bitcoin and nostr. Worse in the misinformation and faith domains. The rest is about the same. So we have a model that is less faithful but knows how to live a healthier life. https://sheet.zoho.com/sheet/open/mz41j09cc640a29ba47729fed784a263c1d08?sheetid=0&range=A1 https://huggingface.co/blog/etemiz/benchmarking-ai-human-alignment-of-grok-3

published an article 5 days ago

Benchmarking Human Alignment of Grok 3

View all activity

Organizations

None yet

etemiz's activity

New activity in mradermacher/Llama-4-Maverick-17B-128E-Instruct-GGUF about 11 hours ago

bug free

#2 opened about 11 hours ago by

etemiz

posted an update 5 days ago

Post

1588

Grok 3 Human Alignment Score: 42

It is better in health, nutrition, fasting compared to Grok 2. About the same in liberating tech like bitcoin and nostr. Worse in the misinformation and faith domains. The rest is about the same. So we have a model that is less faithful but knows how to live a healthier life.

https://sheet.zoho.com/sheet/open/mz41j09cc640a29ba47729fed784a263c1d08?sheetid=0&range=A1

https://huggingface.co/blog/etemiz/benchmarking-ai-human-alignment-of-grok-3

published an article 5 days ago

Article

Benchmarking Human Alignment of Grok 3

•

5 days ago

• 1

replied to Dragunflie-420's post 8 days ago

Have you researched MUDs? It may be easier to code, like doing modifications to a text file. Obviously it won't have graphics but your grandson may use his own imagination!

replied to their post 8 days ago

https://www.reddit.com/r/LocalLLaMA/comments/1jufqbn/qwen3_pull_request_sent_to_llamacpp/

replied to their post 10 days ago

I don't think it is too much random clicking. There is legitimacy to it.

I also think small portion of the data should be public. If any auditor wants, they can get a bigger portion of the data. LLM builders should not get all the data, thats for sure. I will try to do that for my leaderboard, a gradient of openness for different actors.

posted an update 10 days ago

Post

2163

It looks like Llama 4 team gamed the LMArena benchmarks by making their Maverick model output emojis, longer responses and ultra high enthusiasm! Is that ethical or not? They could certainly do a better job by working with teams like llama.cpp, just like Qwen team did with Qwen 3 before releasing the model.

In 2024 I started playing with LLMs just before the release of Llama 3. I think Meta contributed a lot to this field and still contributing. Most LLM fine tuning tools are based on their models and also the inference tool llama.cpp has their name on it. The Llama 4 is fast and maybe not the greatest in real performance but still deserves respect. But my enthusiasm towards Llama models is probably because they rank highest on my AHA Leaderboard:

https://sheet.zoho.com/sheet/open/mz41j09cc640a29ba47729fed784a263c1d08

Looks like they did a worse job compared to Llama 3.1 this time. Llama 3.1 has been on top for a while.

Ranking high on my leaderboard is not correlated to technological progress or parameter size. In fact if LLM training is getting away from human alignment thanks to synthetic datasets or something else (?), it could be easily inversely correlated to technological progress. It seems there is a correlation regarding the location of the builders (in the West or East). Western models are ranking higher. This has become more visible as the leaderboard progressed, in the past there was less correlation. And Europeans seem to be in the middle!

Whether you like positive vibes from AI or not, maybe the times are getting closer where humans may be susceptible to being gamed by an AI? What do you think?

4 replies

liked a model 12 days ago

unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF

Image-Text-to-Text • Updated 10 days ago • 26.9k • 16

posted an update 13 days ago

Post

576

Initial AHA benchmark of Llama 4 Scout puts it in between Command R+ 1 and DeepSeek V3 0324. More numbers later when I do finer benchmark with more updated inference engines.

liked a model 13 days ago

bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-GGUF

Text Generation • Updated 13 days ago • 133k • 29

upvoted an article 15 days ago

Article

Welcome Llama 4 Maverick & Scout on Hugging Face!

16 days ago

• 140

liked a model 16 days ago

unsloth/Llama-4-Scout-17B-16E-Instruct

Image-Text-to-Text • Updated 10 days ago • 9.97k • 56

posted an update 21 days ago

Post

1664

Made a new leaderboard where we measure AI—Human alignment

https://huggingface.co/blog/etemiz/aha-leaderboard

reacted to danielhanchen's post with ❤️ 21 days ago

Post

3346

You can now run DeepSeek-V3-0324 on your own local device!
Run our Dynamic 2.42 and 2.71-bit DeepSeek GGUFs: unsloth/DeepSeek-V3-0324-GGUF

You can run them on llama.cpp and other inference engines. See our guide here: https://docs.unsloth.ai/basics/tutorial-how-to-run-deepseek-v3-0324-locally

published an article 22 days ago

Article

AHA Leaderboard

•

22 days ago

• 1

replied to their post 24 days ago

Here you can find some differences in answers. Enjoy!

https://sheet.zohopublic.com/sheet/published/3b130e0b0581948c04495a434c22958eced33

reacted to luigi12345's post with 👍 24 days ago

Post

3444

🧠 PROMPT FOR CONVERTING ANY MODEL IN REASONING "THINKING" MODEL🔥🤖
Convert any model to Deepseek R1 like "thinking" model. 💭

You're now a thinking-first LLM. For all inputs:

1. Start with <thinking>
   - Break down problems step-by-step
   - Consider multiple approaches
   - Calculate carefully
   - Identify errors
   - Evaluate critically
   - Explore edge cases
   - Check knowledge accuracy
   - Cite sources when possible

2. End with </thinking>

3. Then respond clearly based on your thinking.

The <thinking> section is invisible to users and helps you produce better answers.

For math: show all work and verify
For coding: reason through logic and test edge cases
For facts: verify information and consider reliability
For creative tasks: explore options before deciding
For analysis: examine multiple interpretations

Example:
<thinking>
[Step-by-step analysis]
[Multiple perspectives]
[Self-critique]
[Final conclusion]
</thinking>

[Clear, concise response to user]

4 replies

posted an update 26 days ago

Post

1944

Latest DeepSeek V3 0324 did better than previous version in many domains such as health, nutrition, fasting, bitcoin.

Who wants to see some example change of answers between the two models?

https://sheet.zoho.com/sheet/open/mz41j09cc640a29ba47729fed784a263c1d08

2 replies

posted an update 28 days ago

Post

491

Mistral Small 3.1 numbers are in. It is interesting Mistral always lands in the middle.
https://sheet.zoho.com/sheet/open/mz41j09cc640a29ba47729fed784a263c1d08?sheetid=0&range=A1

I started to do the comparison with 2 models now. In the past Llama 3.1 70B Q4 was the one doing the comparison of answers. Now I am using Gemma 3 27B Q8 as well to have a second opinion on it. Gemma 3 produces very similar measurement to Llama 3.1. So the end result is not going to shake much.

1 reply

replied to their post about 1 month ago

Looks like we need more mature tools for Gemma 3, it is failing to fine tune like half of the time. Unsloth and transformers are getting ready. And I am trying lower learning rates and rank stabilized LoRa, and different r, lora_alpha.