Article
Trainable Dynamic Mask Sparse Attention: Bridging Efficiency and Effectiveness in Long-Context Language Models
By
and 2 others
•
•
5AlbertShi
AlbertShi
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
18 days ago
Trainable Dynamic Mask Sparse Attention
Organizations
upvoted
an
article
9 days ago
upvoted
a
paper
18 days ago
reacted to
JingzeShi's
post with 🤗
18 days ago
Post
3999
Trainable selective sampling and sparse attention kernels are indispensable in the era of context engineering. We hope our work will be helpful to everyone! 🤗
Trainable Dynamic Mask Sparse Attention (2508.02124)
Trainable Dynamic Mask Sparse Attention (2508.02124)
liked
a
dataset
4 months ago
reacted to
JingzeShi's
post with 🚀
4 months ago
Post
2688
@SmallDoge
SmallTalks(
SmallDoge/SmallTalks) is a synthetic dataset designed for supervised fine-tuning of language models. The dataset covers a variety of conversational content, including daily conversations, tool usage, Python programming, encyclopedia Q&A, exam problem-solving, logical reasoning, and more. Each task is provided in both English and Chinese versions.
upvoted
a
collection
5 months ago
liked
3
models
5 months ago
liked
a
dataset
6 months ago
reacted to
JingzeShi's
post with 🚀
6 months ago
Post
4835
We distill a more accurate and concise dataset from DeepSeek R1, and also provide a distillation pipeline code repository.🤗
Dataset: SmallDoge/SmallThoughts
Code: https://github.com/SmallDoges/small-thoughts
Dataset: SmallDoge/SmallThoughts
Code: https://github.com/SmallDoges/small-thoughts
liked
a
model
6 months ago
upvoted
a
paper
7 months ago
liked
4
models
7 months ago
upvoted
a
collection
7 months ago
liked
2
models
7 months ago