muzic

MusicBERT

MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training, by Mingliang Zeng, Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, Tie-Yan Liu, ACL 2021, is a large-scale pre-trained model for symbolic music understanding. It has several mechanisms including OctupleMIDI encoding and bar-level masking strategy that are specifically designed for symbolic music data, and achieves state-of-the-art accuracy on several music understanding tasks, including melody completion, accompaniment suggestion, genre classification, and style classification.

Projects using MusicBERT:

midiformers: a customized MIDI music remixing tool with easy interface for users. (notebook)

Model structure of MusicBERT

OctupleMIDI encoding

1. Preparing datasets

1.1 Pre-training datasets

Prepare The Lakh MIDI Dataset (LMD-full) in zip format for pre-training. (say lmd_full.zip)

wget http://hog.ee.columbia.edu/craffel/lmd/lmd_full.tar.gz
tar -xzvf lmd_full.tar.gz
zip -r lmd_full.zip lmd_full

Run the dataset processing script. (preprocess.py)
```
python -u preprocess.py
```

The script should prompt you to input the path of the midi zip and the path for OctupleMIDI output.

Dataset zip path: /xxx/xxx/MusicBERT/lmd_full.zip
OctupleMIDI output path: lmd_full_data_raw
SUCCESS: lmd_full/a/0000.mid
......

Binarize the raw text format dataset. (this script will read lmd_full_data_raw folder and output lmd_full_data_bin)
```
bash binarize_pretrain.sh lmd_full
```

1.2 Melody completion and accompaniment suggestion datasets

Follow “PiRhDy: Learning Pitch-, Rhythm-, and Dynamics-aware Embeddings for Symbolic Music” (https://github.com/mengshor/PiRhDy) to generate datasets for melody completion task and accompaniment suggestion task, or download generated datasets directly.
```
PiRhDy/dataset/context_next/train
PiRhDy/dataset/context_next/test
PiRhDy/dataset/context_acc/train
PiRhDy/dataset/context_acc/test
```
Convert these two datasets to OctupleMIDI format with gen_nsp.py.
```
python -u gen_nsp.py
```
The script should prompt you to input which downstream task to process. (next for melody task and acc for accompaniment task)
```
task = next
```
Binarize the raw text format dataset. (this script will read next_data_raw folder and output next_data_bin)
```
bash binarize_nsp.sh next
```

1.3 Genre and style classification datasets

Prepare The Lakh MIDI Dataset (LMD-full) in zip format. (say lmd_full.zip, ignore this step if you have lmd_full.zip)

wget http://hog.ee.columbia.edu/craffel/lmd/lmd_full.tar.gz
tar -xzvf lmd_full.tar.gz
zip -r lmd_full.zip lmd_full

Get TOPMAGD and MASD midi to genre mapping midi_genre_map from “On large-scale genre classification in symbolically encoded music by automatic identification of repeating patterns”.(DLfM 2018) (https://github.com/andrebola/patterns-genres)
```
wget https://raw.githubusercontent.com/andrebola/patterns-genres/master/data/midi_genre_map.json
```
Generate these two datasets in OctupleMIDI format using the midi to genre mapping file with gen_genre.py.
```
python -u gen_genre.py
```
The script should prompt you to input which downstream task to process. (topmagd for genre task and masd for style task)
```
subset: topmagd
LMD dataset zip path: lmd_full.zip
sequence length: 1000
```
Binarize the raw text format dataset. (this script will read topmagd_data_raw folder and output topmagd_data_bin)
```
bash binarize_genre.sh topmagd
```

2. Training

2.1 Pre-training

bash train_mask.sh lmd_full small

Download our pre-trained checkpoints here: small and base, and save in the ` checkpoints` folder. (a newer version of fairseq is needed for using provided checkpoints: see issue-37 or issue-45)

2.2 Fine-tuning on melody completion task and accompaniment suggestion task

bash train_nsp.sh next checkpoints/checkpoint_last_musicbert_base.pt

bash train_nsp.sh acc checkpoints/checkpoint_last_musicbert_small.pt

2.3 Fine-tuning on genre and style classification task

bash train_genre.sh topmagd 13 0 checkpoints/checkpoint_last_musicbert_base.pt

bash train_genre.sh masd 25 4 checkpoints/checkpoint_last_musicbert_small.pt

3. Evaluation

3.1 Melody completion task and accompaniment suggestion task

python -u eval_nsp.py checkpoints/checkpoint_last_nsp_next_checkpoint_last_musicbert_base.pt next_data_bin

3.2 Genre and style classification task

python -u eval_genre.py checkpoints/checkpoint_last_genre_topmagd_x_checkpoint_last_musicbert_small.pt topmagd_data_bin/x

This site is open source. Improve this page.