Article
Leveraging Hugging Face for complex generative AI use cases
By
and 1 other
•
Waseem AlShikh
wassemgtk
AI & ML interests
Multi-modal, Palmyra LLMs, Knowledge Graph
Recent Activity
updated
a model
11 days ago
wassemgtk/mergekit-ties-isswcgh
published
a model
11 days ago
wassemgtk/mergekit-ties-isswcgh
replied to
their
post
14 days ago
I’ve been diving into the iRoPE architecture from Llama 4—a game-changer for long-context models! It interleaves local attention (with RoPE) for short contexts and global attention (with inference-time temp scaling) for long-range reasoning, aiming for infinite context. I’m going to try writing iRoPE—who wants to help?
Code: https://github.com/wassemgtk/iRoPE-try/blob/main/iRoPE.ipynb
Organizations
