Steve Welburn

Contact Details

Title: Research Student
Tel: National: 020 7882 7986
International: +44 20 7882 7986
Fax: National: 020 7882 7997
International: +44 20 7882 7997
Email: stephen.welburn@elec.qmul.ac.uk
Room: 109

Research Lab: Centre for Digital Music

Supervisor: Mark Plumbley

Research Topic: Application of Machine Learning Techniques to the Sparse Object-Based Coding of Music

The aim of this project is to find an object-based coding representation for musical signals which combines efficient encoding/decoding algorithms with a high-quality approximation of the signal. A trade-off is expected between the quality of the representation and the speed of the coding, and we will be examining techniques to parameterise this trade-off.

Object-based coding combines a set of predictable objects to produce a more complex behaviour. Each of the objects encapsulates a basic behaviour pattern (although this behaviour may be parameter dependent). For an audio signal, we wish to represent the evolution of the signal over time as a combination of some underlying time-evolving objects and a series of time-dependent parameters.

The most well-known "object" in audio signal processing is the sinusoid, and the FFT effectively represents the time-based signal as a collection of sinusoidal objects, each representing a given frequency. However, this representation contains many objects, some of which are interdependent (e.g. the harmonics of a note). For efficient coding, we want to find a coding in which few objects will be active at once - e.g. a sparse coding.

For music, the most well-known sparse coding is a score - indicating which instruments are playing when (and pitches, durations and dynamics). This is, however, only an approximation of a performance as (for example) performers musical expression, and room acoustics affect the actual signal produced.

The problem of sparse object-based coding of music is therefore closely related to that of automatic music transcription. However, unlike transcription, we are not interested in inferring the original physical sources of the signals, rather we only wish to represent the signal itself. It may be that the coding is most efficient when the physical sources are individually represented, but if a more compact representation can be found then that will be our preference.

Techniques developed for music transcription (e.g. pitch estimation and onset detection) allow us to extract a probability distributions for significant features from a signal. Having extracted pdfs for features, we will look to build a probability distribution for the active objects in the signal. According to Bayes Rule, we can update these distributions according to prior knowledge, which we will base on music theory (e.g. using estimates of both the chords in the music and the underlying key we will update our distribution according to how likely the notes are).

In addition to harmonic objects, we will look at classifying sources of noise in the signal (e.g. breathing, hammer action) and will aim to develop suitable objects for various types of noise.

By encapsulating models for the different types of sounds in objects and by allowing these objects to evolve with the signal, we aim to be able learn the most appropriate objects to represent a given signal. By restricting the learning techniques applied during coding to those suitable for online-learning (i.e. processing the signal-to-date at a point in time, rather than the "complete" signal) we intend that the coding model developed will be suitable for subsequent application in audio streaming, and online / real-time coding. Initial learning of our base library of objects may be accomplished using a broader range of machine leanring techniques (supervised learning, offline learning)

By combining learnt harmonic objects and noise objects with a representation of any residual signal, we aim to develop an efficient coding technique for the representation of music and develop a base object library which will allow new signals to be compactly represented.