jinjieyuan commited on
Commit
49c4558
·
verified ·
1 Parent(s): f8b7eab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -5
README.md CHANGED
@@ -3,9 +3,34 @@ language: en
3
  license: apache-2.0
4
  ---
5
 
6
- # Mamba-Shedder Model: mamba-shedder-mamba2-2.7b-ssm22
7
 
8
- | Source Model | Components Removed | Recovery Tuning | WikiText PPL | Avg. Score | Relative Acc. |
9
- |--------------------------------------------------------------------|-----------------------------------------|-----------------|---------------|------|------|
10
- | [mamba2-2.7b](https://huggingface.co/state-spaces/mamba2-2.7b) | - | - | 4.10 | 60.2 | 100% |
11
- | [mamba-shedder-mamba2-2.7b-ssm22](https://huggingface.co/IntelLabs/mamba-shedder-mamba2-2.7b-ssm22) | 22 SSDs | ✘ | 6.50 | 57.8 | 96% |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  license: apache-2.0
4
  ---
5
 
6
+ # Mamba-Shedder Model: Mamba-Shedder-Mamba-2.2B
7
 
8
+ - Base Model: [state-spaces/mamba2-2.7b](https://huggingface.co/state-spaces/mamba2-2.7b)
9
+ - Pruned Components: **22 SSDs** (Layer 63, 54, 42, 45, 53, 57, 58, 59, 38, 56, 50, 61, 60, 43, 37, 62, 49, 34, 55, 33, 39, 35)
10
+ - Recovery Tuning: No
11
+
12
+ ### Evaluation
13
+
14
+ ```bash
15
+ git clone https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning.git
16
+ cd Mamba-Shedder
17
+ python eval.py --model_path <path to model>
18
+ ```
19
+
20
+ Refer to our [code repository](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder) for the environment information to run this command.
21
+
22
+ ## Model Sources
23
+
24
+ - **Repository:** [https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder](https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning/tree/main/Mamba-Shedder)
25
+ - **Paper:** [Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models]()
26
+
27
+ ## Citation
28
+
29
+ ```bibtex
30
+ @article{munoz2025mambashedder,
31
+ title = {Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models},
32
+ author = {J. Pablo Munoz and Jinjie Yuan and Nilesh Jain},
33
+ journal = {},
34
+ year = {2025}
35
+ }
36
+ ```