DJ-Agent: music theory directed a cappella accompaniment generation using deep reinforcement learning

Jiuming Jiang

doi:10.1117/12.2667902

28 March 2023 DJ-Agent: music theory directed a cappella accompaniment generation using deep reinforcement learning

Jiuming Jiang

Proceedings Volume 12566, Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022); 125663S (2023) https://doi.org/10.1117/12.2667902
Event: Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022), 2022, Chongqing, China

Abstract

This paper proposes a song accompaniment generation method that combines audio analysis and symbolic music generation so that human music theory can be used to build a reinforcement learning model, training an agent to create music. The key to this algorithm is to extract music theory concepts from audio and a reward model that works well in reinforcement learning. However, some music theory rules are complex and challenging to describe. It is difficult to achieve competitive results only by hardcoding the reward. Therefore, to build an effective reward model, a neural network is used to evaluate the perceptual part of composition quality, and program discrimination is used to model easy-to-describe music theory, and the two work together. Experiments show that the proposed algorithm can generate accompaniment arrangements close to human composers, is compatible with various musical styles, and outperforms the baseline algorithm in multiple evaluation metrics.

Citation Download Citation

Jiuming Jiang "DJ-Agent: music theory directed a cappella accompaniment generation using deep reinforcement learning", Proc. SPIE 12566, Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022), 125663S (28 March 2023); https://doi.org/10.1117/12.2667902

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available