RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning


  • Nan Jiang Tsinghua University
  • Sheng Jin Tsinghua University
  • Zhiyao Duan Unversity of Rochester
  • Changshui Zhang Tsinghua University




This paper presents a deep reinforcement learning algorithm for online accompaniment generation, with potential for real-time interactive human-machine duet improvisation. Different from offline music generation and harmonization, online music accompaniment requires the algorithm to respond to human input and generate the machine counterpart in a sequential order. We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state). The key of this algorithm is the well-functioning reward model. Instead of defining it using music composition rules, we learn this model from monophonic and polyphonic training data. This model considers the compatibility of the machine-generated note with both the machine-generated context and the human-generated context. Experiments show that this algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part. Subjective evaluations on preferences show that the proposed algorithm generates music pieces of higher quality than the baseline method.




How to Cite

Jiang, N., Jin, S., Duan, Z., & Zhang, C. (2020). RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 710-718. https://doi.org/10.1609/aaai.v34i01.5413



AAAI Technical Track: Applications