Research Seminars

Instrument models for source separation and transcription of music recordings

Emmanuel Vincent, Centre for Digital Music at Queen Mary, University of London

Wednesday 17 of November at 11.30am

Abstract

For about fifteen years the study of ensemble music recordings has focused on two distinct viewpoints: source separation and polyphonic transcription. Source separation tries to extract from a recording the signals corresponding to each musical instrument playing. Polyphonic transcription aims at describing a recording by a set of parameters: names of the instruments, pitch and loudness of the notes, etc. Existing methods, based on spatial and spectro-temporal analysis of the recordings, provide satisfying results in simple cases. But their performance generally degrades fast with more instruments than a fixed limit, under reverberant conditions, with instruments of similar pitch ranges or with notes at harmonic intervals.

In this talk, we show that these problems can be reduced using specific models of instrumental sources trained on solo data. We design probabilistic instrument models inspired from Independent Subspace Analysis (ISA) that take into account harmonicity, timbre and temporal evolution. Then we exploit these models to separate and transcribe realistic recordings, among which a duo CD track and synthetic convolutive or underdetermined mixtures of solo CD tracks.