Hugging Face's logo
tim-lawson 's Collections
Learning to Skip the Middle Layers of Transformers
Multi-Layer SAEs
Multi-Layer SAEs with Tuned Lens
Multi-Layer SAEs with Transformers
Multi-Layer SAEs with Tuned Lens and Transformers
Single-Layer SAEs
Single-Layer SAEs with Transformers

Multi-Layer SAEs

Single SAEs trained on the residual stream activation vectors from every transformer layer simultaneously: https://arxiv.org/abs/2409.04185

Upvote
-