Research Seminars

Active Music Listening Interfaces Based on Music-Understanding Technologies

Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST), Japan

30 June 2008, 14:00, Room 105

Abstract

People who can actively interact with music have traditionally been considered musicians in the sense that ``actively'' implies the creation of music. On the other hand, ordinary people have been just listeners who could interact with music only passively. When the recording of music to audio storage media became a reality, however, some people started interacting with music in more active ways, for example, by specifying the playback order of songs or adjusting frequency characteristics by using tone controls. Recent advances in computer and music-understanding technologies will further affect how people interact with music.

In this talk, I will introduce our research aimed at building ``Active Music Listening Interfaces'' [M. Goto, Proc. of IEEE ICASSP 2007] to demonstrate the importance of music-understanding technologies and the benefit they offer to ordinary people (end users). Given polyphonic sound mixtures taken from available music recordings, our interfaces enrich end-users' music listening experiences by applying our automatic music-understanding technologies based on signal processing. In this research, ``active'' does not mean the creation of new music, but any active experience that is part of enjoying music.

For example, our active music listening interface with a chorus-search function, "SmartMusicKIOSK" [M. Goto, IEEE Trans. ASLP, Vol.14, No.5, 2006], enables a user to skim rapidly through a musical piece by easily skipping sections of no interest while viewing a visual representation of music structure. During the playback of a song, "LyricSynchronizer" [H. Fujihara et al., Proc. of IEEE ISM 2006] with a lyrics synchronization function displays scrolling lyrics and highlights the phrase currently sung. A user can easily follow the current playback position and click on a word in the lyrics to listen to it. By suppressing drum sounds and adding other drum sounds, "Drumix" [K. Yoshii et al., IPSJ Journal, Vol.48, No.3, 2007] with a drum-sound recognition function enables a user to change the volume and timbre of drum sounds and rearrange rhythmic patterns of these drum sounds during playback.

Bio

Masataka Goto received the Doctor of Engineering degree from Waseda University, Japan, in 1998. He is a Senior Research Scientist of the National Institute of Advanced Industrial Science and Technology (AIST), Japan. He serves concurrently as an Associate Professor of the University of Tsukuba since 2005 and a Visiting Associate Professor of The Institute of Statistical Mathematics since 2008. He received 22 awards, including the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology "Young Scientists' Prize" in 2008, DoCoMo Mobile Science Awards "Excellence Award in Fundamental Science" in 2007, and IPSJ (The Information Processing Society of Japan) Best Paper Award in 2005.