Andrew Robertson

Contact Details

Title: Research Student
Tel: National: 020 7882 7986
International: +44 20 7882 7986
Fax: National: 020 7882 7997
International: +44 20 7882 7997
Email: andrew.robertson@elec.qmul.ac.uk
Room: 109

Research Lab: Centre for Digital Music

Research Topic: Interactive Real-time Musical Systems

The development of human-computer interactive systems has been a long-term goal for many within the audio processing community. At present there are many techniques available for audio analysis, although their implementation within real-time interactive systems has yet to be achieved.

In a related problem, human-to-human interaction over long-distance networks involves necessary delays that present considerable difficulties for musical interactivity over such networks. These two modes of performance require both the monitoring and negotiating of these delays in order to have successful musical interaction.

This project aims to investigate automatic interactive systems, both computer-mediated automatic accompaniment and performance across a network. In the first case, an interactive system would perform successful score-following and automatic accompaniment, whilst monitoring the delays inherent in the system, so as to produce a predictive output that is synchronous with the human performance.

In the case of performances across a network, the transmission of pseudo-performances might be used to mediate the inherent delay. In such a system, The performers might be responding to a previous performance, recorded during a training period, which is sent early so as to synchronize with the performers at the other end of the network. The system would need to learn the nuances of the performance during a training period and be able to modify the audio signal to allow successful real-time interactive performance over the network.

Both of these interactive systems involve the construction of real-time predictive demonstrator systems in which the audio from a performance is related to a representation of the musical structure of the piece. The computation required takes place after the actual musical events and hence the output of the system would need to be predictive as well as reactive.

The techniques used would include many of the audio analysis tools that have presently been developed: onset detection, beat/tempo tracking, harmonic progression modeling, automatic pitch detection and chordal/chroma segmentation. Much of the work has been completed for these tools but they need to be implemented to run in real-time with a reliable delay estimation. A successful human-computer interactive system would construct a probabilstic model of the piece, allowing prediction of future events, so that real-time interactivity is achieved. The probabilistic methods under consideration involve Hidden Markov models, Bayesian Networks and time-delay neural networks. This project will incorporate many of the audio analysis techniques available into a unified system which, by operating in real-time, could create new modes of musical performance.

Publications

A. Robertson and M. D. Plumbley. Real-time beat-tracker for live performance with drums. To appear in Proceedings of the Digital Music Research Network Summer Conference, Leeds Metropolitan University, UK, 7-8 July 2007.

A. Robertson and M. D. Plumbley. B-Keeper: A beat tracker for real time synchronisation within performance. In Proceedings of New Interfaces for Musical Expression (NIME 2007), New York, NY, USA, June 6-10, 2007, pp 234-237, 2007.

A. N. Robertson and M. D. Plumbley. Real-time Interactive Musical Systems: An Overview. In Proceedings of the Digital Music Research Network Doctoral Researchers Conference, London, UK, July 22-23, 2006.