File size: 5,023 Bytes
d04f31f 4068b97 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 |
---
title: ZFTurbo Web-UI
emoji: 🎵
colorFrom: '#D00000'
colorTo: '#FFBA08'
sdk: gradio
sdk_version: '3.27.0'
app_file: app.py
pinned: false
---
# MVSEP-MDX23-music-separation-model
Model for [Sound demixing challenge 2023: Music Demixing Track - MDX'23](https://www.aicrowd.com/challenges/sound-demixing-challenge-2023). Model perform separation of music into 4 stems "bass", "drums", "vocals", "other". Model won 3rd place in challenge (Leaderboard C).
Model based on [Demucs4](https://github.com/facebookresearch/demucs), [MDX](https://github.com/kuielab/mdx-net) neural net architectures and some MDX weights from [Ultimate Vocal Remover](https://github.com/Anjok07/ultimatevocalremovergui) project (thanks [Kimberley Jensen](https://github.com/KimberleyJensen) for great high quality vocal models). Brought to you by [MVSep.com](https://mvsep.com).
## Usage
```
python inference.py --input_audio mixture1.wav mixture2.wav --output_folder ./results/
```
With this command audios with names "mixture1.wav" and "mixture2.wav" will be processed and results will be stored in `./results/` folder in WAV format.
### All available keys
* `--input_audio` - input audio location. You can provide multiple files at once. **Required**
* `--output_folder` - output audio folder. **Required**
* `--cpu` - choose CPU instead of GPU for processing. Can be very slow.
* `--overlap_large` - overlap of splitted audio for light models. Closer to 1.0 - slower, but better quality. Default: 0.6.
* `--overlap_small` - overlap of splitted audio for heavy models. Closer to 1.0 - slower, but better quality. Default: 0.5.
* `--single_onnx` - only use single ONNX model for vocals. Can be useful if you have not enough GPU memory.
* `--chunk_size` - chunk size for ONNX models. Set lower to reduce GPU memory consumption. Default: 1000000.
* `--large_gpu` - it will store all models on GPU for faster processing of multiple audio files. Requires at least 11 GB of free GPU memory.
* `--use_kim_model_1` - use first version of Kim model (as it was on contest).
* `--only_vocals` - only create vocals and instrumental. Skip bass, drums, other. Processing will be faster.
### Notes
* If you have not enough GPU memory you can use CPU (`--cpu`), but it will be slow. Additionally you can use single ONNX (`--single_onnx`), but it will decrease quality a little bit. Also reduce of chunk size can help (`--chunk_size 200000`).
* In current revision code requires less GPU memory, but it process multiple files slower. If you want old fast method use argument `--large_gpu`. It will require > 11 GB of GPU memory, but will work faster.
* There is [Google.Collab version](https://colab.research.google.com/github/jarredou/MVSEP-MDX23-Colab_v2/blob/main/MVSep-MDX23-Colab.ipynb) of this code.
## Quality comparison
Quality comparison with best separation models performed on [MultiSong Dataset](https://mvsep.com/quality_checker/leaderboard2.php?sort=bass).
| Algorithm | SDR bass | SDR drums | SDR other | SDR vocals | SDR instrumental |
| ------------- |:---------:|:----------:|:----------:|:----------:|:------------------:|
| MVSEP MDX23 | 12.5034 | 11.6870 | 6.5378 | 9.5138 | 15.8213 |
| Demucs HT 4 | 12.1006 | 11.3037 | 5.7728 | 8.3555 | 13.9902 |
| Demucs 3 | 10.6947 | 10.2744 | 5.3580 | 8.1335 | 14.4409 |
| MDX B | --- | ---- | --- | 8.5118 | 14.8192 |
* Note: SDR - signal to distortion ratio. Larger is better.
## GUI
* Script for GUI (based on PyQt5): [gui.py](gui.py).
* You can download [standalone program for Windows here](https://github.com/ZFTurbo/MVSEP-MDX23-music-separation-model/releases/download/v1.0.1/MVSep-MDX23_v1.0.1.zip) (~730 MB). Unzip archive and to start program double click `run.bat`. On first run it will download pytorch with CUDA support (~2.8 GB) and some Neural Net models.
* Program will download all needed neural net models from internet at the first run.
* GUI supports Drag & Drop of multiple files.
* Progress bar available.
## Changes
### v1.0.1
* Settings in GUI updated, now you can control all possible options
* Kim vocal model updated from version 1 to version 2, you still can use version 1 using parameter `--use_kim_model_1`
* Added possibility to generate only vocals/instrumental pair if you don't need bass, drums and other stems. Use parameter `--only_vocals`
* Standalone program was updated. It has less size now. GUI will download torch/cuda on the first run.
## Citation
* [arxiv paper](https://arxiv.org/abs/2305.07489)
```
@misc{solovyev2023benchmarks,
title={Benchmarks and leaderboards for sound demixing tasks},
author={Roman Solovyev and Alexander Stempkovskiy and Tatiana Habruseva},
year={2023},
eprint={2305.07489},
archivePrefix={arXiv},
primaryClass={cs.SD}
}
``` |