Jump to the main content block
TITLE

[Oct-09] Learning to Generate Jazz and Pop Piano Music from Audio via MIR Techniques

Seminar of Institute of Information Systems and Applications

Speaker :

Dr. Yi-Hsuan Yang

Associate Research Fellow of the Research Center for IT Innovation, Academia Sinica.

Topic :

Learning to Generate Jazz and Pop Piano Music from Audio via MIR Techniques

Date :

13:30-15:00 Wednesday 9-Oct-2019

Location :

台達館R105Delta Building R105

Hosted by:

Prof. Chun-Yi Lee

Abstract

We explore at the Taiwan AI Labs a new approach that learns to compose music from musical audio recordings, by capitalizing state-of-the-art music information retrieval (MIR) techniques. Given an input audio recording, we firstly apply blind source separation to isolate out (from the input) the individual musical sources of interest, such as the piano, drums, bass, and vocal. We then apply music transcription to convert the separated sources (which are audio files) into the symbolic domain by predicting the pitch, onset/offset time (in absolute timing), and velocity (dynamics) of the involved notes. In the meanwhile, we perform beat and downbeat detection over the input audio to get the underlying metrical grid of the music. We then use the machine-transcribed symbolic data, possibly along with other machine-predicted information (such as genre/mood tags, musical key, tempo, chords, and the use of playing techniques) to train our music composition model. The music generated by our composition model is finally rendered into audio with a synthesizer. As our composition model is supplied with performance-level information such as variations in velocity and microtimings (timing offsets) that is present in the audio files, it has the promise to generate expressive music.

Bio

Dr. Yi-Hsuan Yang is currently with the Taiwan AI Labs as the Chief Music Scientist leading a team working on music AI technologies.  He also holds a position as an Associate Research Fellow of the Research Center for IT Innovation, Academia Sinica. He received his Ph.D. degree in Communication Engineering from National Taiwan University in 2010. His research interests include music information retrieval, artificial intelligence, affective computing, and machine learning. Dr. Yang was a recipient of the 2011 IEEE Signal Processing Society Young Author Best Paper Award, the 2012 ACM Multimedia Grand Challenge First Prize, the 2014 Ta-You Wu Memorial Research Award of the Ministry of Science and Technology, Taiwan, the 2015 Young Scholars’ Creativity Award from the Foundation for the Advancement of Outstanding Scholarship, and the 2019 Multimedia Rising Stars Award from the IEEE International Conference on Multimedia Expo. He is an author of the book Music Emotion Recognition (CRC Press 2011). He was a Technical Program Co-Chair of the International Society for Music Information Retrieval Conference (ISMIR) in 2014. He was an Associate Editor for the IEEE Transactions on Affective Computing and the IEEE Transactions on Multimedia, both from 2016 to 2019. Dr. Yang is a senior member of the IEEE.

All faculties and students are welcome to join.

 

Click Num: