Update README.md
Browse files
README.md
CHANGED
@@ -65,20 +65,7 @@ For all the 4 sizes of SpeechLMM 1.0, the audio and video adapters are:
|
|
65 |
|
66 |
Currently, this model can only be used via our [`speechlmm`](https://github.com/meetween/speechlmm) codebase. Refer to the instructions there for more details.
|
67 |
|
68 |
-
Important: before you can use this model, you must
|
69 |
-
1. Download the SeamlessM4T v2 speech encoder weights:
|
70 |
-
```python
|
71 |
-
from transformers import AutoProcessor, SeamlessM4Tv2Model
|
72 |
-
|
73 |
-
processor = AutoProcessor.from_pretrained("facebook/seamless-m4t-v2-large")
|
74 |
-
model = AutoModel.from_pretrained("facebook/seamless-m4t-v2-large")
|
75 |
-
|
76 |
-
processor.save_pretrained("path/to/some_directory_1")
|
77 |
-
model.speech_encoder.save_pretrained("path/to/some_directory_1")
|
78 |
-
```
|
79 |
-
2. Go to `config.json` and change the `audio_encoder._name_or_path` to `path/to/some_directory_1`
|
80 |
-
3. Download the Auto-AVSR video encoder weights from [here](https://drive.google.com/file/d/1shcWXUK2iauRhW9NbwCc25FjU1CoMm8i/view?usp=sharing) and put them in `path/to/some_directory_2`
|
81 |
-
4. Go to `config.json` and change the `video_encoder._name_or_path` to `path/to/some_directory_2/vsr_trlrs3vox2_base.pth`
|
82 |
|
83 |
## Training Data
|
84 |
|
|
|
65 |
|
66 |
Currently, this model can only be used via our [`speechlmm`](https://github.com/meetween/speechlmm) codebase. Refer to the instructions there for more details.
|
67 |
|
68 |
+
Important: before you can use this model, you must download the SeamlessM4T v2 speech encoder by following the instructions provided in the README of the above repo. Please note that by downloading SeamlessM4Tv2, you agree with its license terms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
69 |
|
70 |
## Training Data
|
71 |
|