muzic
MeloForm
MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks, by Peiling Lu, Xu Tan, Botao Yu, Tao Qin, Sheng Zhao, Tie-Yan Liu, ISMIR 2022, is a melody generation system that leverages expert system and neural networks to generate melodies with musical form.
The overall architecture of our MeloForm framework.
The expert system of MeloForm.
The neural network of MeloForm.
1. Data
We obtain LMD dataset from here and process them to be monophonic. We privode a script to parse LMD data in our experiments. We give some raw data in melody tracks in here as examples to run scripts.
python preprocess_lmd.py ./data/train/raw ./data/train/processed/para
Based on the above scripts, data samples will be generated under the ./data/train/processed/para directory.
We have provide the script to generate binarized data. The script to run is as:
data_dir=./data/train/processed/
user_dir=meloformt
bash binarize.sh $data_dir $user_dir
2. Training
We provide an example script for running.
data_dir=./data/train/processed/processed_para/ # The path of binarized data
user_dir=meloform
bash train.sh $data_dir $user_dir
3. Refinement
For refinement, we need to firstly process sythetic data from expert systems, then neural networks will refine it phrase by phrase iteratively. We provide several melodies from expert system to help test, more data will be released soon.
Firstly, run the following script to process sythetic data from expert systems:
data_dir=./data/refine/expert_system
song_id=0
out_dir=./data/refine/data_nn
python process_es.py $data_dir $song_id $out_dir
The processed results are under ./data/refine/data_nn/{song_id}, in ./data/refine/data_nn/{song_id}/template.
Finally, run
data_dir=./data/refine/data_nn
song_id=0
model_path=checkpoints/
res_dir=results/
bash meloform_refine_melody.sh $data_dir $song_id $model_path $res_dir
The final refined melody will be located in {res_dir}/out_midi/{song_id}/b1/src_res.mid
Notice that we only release the code of neural networks this time, the melody data from expert systems will be released soon.