Research Seminars

Limitations of the bag-of-frame approach to the pattern recognition of musical signals

Jean-Julien Aucouturier, SOAS, University of London

Wednesday 10 May 2006, 4:00pm, Room 207

Abstract

The majority of systems extracting high-level music descriptions from audio signals rely on a common, implicit model of their global sound or "polyphonic timbre". This model represents the timbre of a texture as the long-term distribution of its local spectral features. The underlying assumption is rarely made explicit: the perception of the timbre of a texture is assumed to result from the most statistically significant feature windows. This talk questions the validity of this assumption. To do so, we construct an explicit measure of the timbre similarity between polyphonic music textures, and variants thereof inspired by previous work in Music Information Retrieval. We show that such algorithms tend to create false positives ("hubs") which are mostly always the same songs regardless of the query, i.e. songs that are irrelevantly close to every other songs. Their study suggests that the perceptual saliency of feature observations is not necessarily correlated with their statistical significance with respect to the global distribution. In other words, music listeners routinely "hear'' things that are not statistically significant in musical signals, but rather are the result of high-level cognitive reasoning, which depends on cultural expectations, a priori knowledge, and context.

Bio

Jean-Julien Aucouturier (jj@csl.sony.fr) was until recently an assistant researcher at Sony Computer Science Laboratory, Paris (France), while studying for the PhD in Artificial Intelligence at University of Paris 6 (expected June 2006). He received a MSc. in Music Signal Processing from King's College, University of London and graduated as an electronic engineer from Ecole Supérieure d'Electricité, Rennes (France). He is currently an invited research fellow in Ethnomusicology at SOAS, University of London, working on cognitive models of embodied perception of music.