File size: 2,469 Bytes
f8b7eab c500a06 f8b7eab 49c4558 c09fd92 49c4558 010fe89 49c4558 30afec5 49c4558 010fe89 49c4558 5d56e2f 010fe89 49c4558 010fe89 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
---
language: en
license: apache-2.0
---
# Mamba-Shedder Model: Mamba-Shedder-Mamba2-2.7B-Pruned-22SSM
- Base Model: [state-spaces/mamba2-2.7b](https://huggingface.co/state-spaces/mamba2-2.7b)
- Pruned Components: **22 SSMs** (Layer 63, 54, 42, 45, 53, 57, 58, 59, 38, 56, 50, 61, 60, 43, 37, 62, 49, 34, 55, 33, 39, 35)
- Recovery Tuning: No
### Evaluation
```bash
git clone https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning.git
cd Mamba-Shedder
python eval.py --model_path <path to model>
```
Refer to our [code repository](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder) for the environment information to run this command.
## Ethical Considerations
Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See [Intel’s Global Human Rights Principles](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf). Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.
## Model Sources
- **Repository:** [https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder)
- **Paper:** [Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models](https://arxiv.org/abs/2501.17088)
## Citation
```bibtex
@inproceedings{munoz2025mambashedder,
title = {Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models},
author = {Mu{\~n}oz, J. Pablo and Yuan, Jinjie and Jain, Nilesh},
booktitle = "Proceedings of the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025)",
month = jun,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "",
}
```
### Original Work Citation
This work builds upon work done by the State-Spaces team. Please see the following for additional citations of their work:
**Repository:** ([state-spaces/mamba](https://github.com/state-spaces/mamba)
**Paper:** [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://arxiv.org/abs/2312.00752)
|