Sigma: Differential Rescaling of Query, Key and Value for Efficient Language Models
Paper
•
2501.13629
•
Published
•
48
Fredi PRO
Fredithefish
AI & ML interests
LLMs, Audio, Diffusion Models
Recent Activity
updated
a model
about 9 hours ago
Fredithefish/newhtmlmodel
published
a model
about 9 hours ago
Fredithefish/newhtmlmodel
liked
a model
1 day ago
microsoft/MAI-DS-R1
Organizations
Fredithefish's activity
upvoted
a
paper
2 months ago
upvoted
an
article
6 months ago
Article
Fixing Gradient Accumulation
•
53
upvoted
5
papers
7 months ago
Self-Rewarding Language Models
Paper
•
2401.10020
•
Published
•
148
Law of the Weakest Link: Cross Capabilities of Large Language Models
Paper
•
2409.19951
•
Published
•
55
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
Paper
•
2410.02367
•
Published
•
50
RATIONALYST: Pre-training Process-Supervision for Improving Reasoning
Paper
•
2410.01044
•
Published
•
37
VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
Paper
•
2410.01679
•
Published
•
25
upvoted
a
paper
9 months ago