Wendy
wwymak
AI & ML interests
None yet
Recent Activity
liked
a dataset
9 days ago
ai4privacy/pii-masking-200k
updated
a collection
16 days ago
small-but-mighty-llms
liked
a model
16 days ago
knowledgator/comprehend_it-base
Organizations
-
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Paper • 2301.09626 • Published • 2 -
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Paper • 2309.04679 • Published -
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Generative LLM Inference
Paper • 2402.10712 • Published -
FOCUS: Effective Embedding Initialization for Specializing Pretrained Multilingual Models on a Single Language
Paper • 2305.14481 • Published • 2
-
TransformerFAM: Feedback attention is working memory
Paper • 2404.09173 • Published • 44 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 14 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
Paper • 2309.10400 • Published • 26 -
winglian/Llama-3-8b-64k-PoSE
Text Generation • 8B • Updated • 89 • • 76 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Paper • 2301.09626 • Published • 2 -
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
Paper • 2309.04679 • Published -
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Generative LLM Inference
Paper • 2402.10712 • Published -
FOCUS: Effective Embedding Initialization for Specializing Pretrained Multilingual Models on a Single Language
Paper • 2305.14481 • Published • 2
-
TransformerFAM: Feedback attention is working memory
Paper • 2404.09173 • Published • 44 -
Ring Attention with Blockwise Transformers for Near-Infinite Context
Paper • 2310.01889 • Published • 14 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68
-
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
Paper • 2309.10400 • Published • 26 -
winglian/Llama-3-8b-64k-PoSE
Text Generation • 8B • Updated • 89 • • 76 -
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
Paper • 2404.08801 • Published • 68