Automatic Polyphonic Music Transcription using Multiple Cause Models and Independent Component Analysis

Contents of this page: Summary | Publications | Software | Participants

Summary

Automatic music transcription, the task of automatically extracting the notes in a piece of music, is particularly difficult for polyphonic music, where more than one note is played at a time. Many approaches have previously been tried using knowledge about musical audio or human hearing. The aim of this project has been to develop new fundamental methods to tackle the automatic music transcription problem, that learn the characteristics of notes from the data, at the same time as extracting the notes themselves. These techniques are based on assumptions that (a) the notes are relatively “independent” from each other, and (b) that the occurrence of notes is “sparse”, i.e. that few notes are present at any one time. The techniques we have been using are known as independent component analysis (ICA) and sparse coding.

In the project, we developed a new method for automatic music transcription based on these techniques. We found that notes were typically represented by groups of a handful of vectors each, representing the way the frequency spectrum of the note changed as the note sounded. We also developed new non-negative ICA and non-negative sparse coding techniques, taking advantage of the fact that note activities must be positive or zero, and investigated a new shift-invariant sparse coding method to learn the waveforms directly. We also used ICA to find independent “basis vectors” to represent music and speech, and developed a method to visualize the relationships between these basis vectors. The visualization for music is reminiscent of the idea of a “circle of fifths” familiar to musicians. We also developed an onset detection method based on ICA, giving a “surprise” signal which is high when a new note begins.

In the future, techniques like this could be used to analyse the content of the huge collections of music in digital formats like MP3. Using this analysis, it would be possible to search through personal or commercial collections of music just as easily as an internet search engine can be used today.

Publications

S. A. Abdallah and M. D. Plumbley: Unsupervised analysis of polyphonic music using sparse coding in a probabilistic framework. To appear in IEEE Transactions on Neural Networks.

S. A. Abdallah and M. D. Plumbley. Polyphonic transcription by non-negative sparse coding of power spectra. In Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR 2004), pp 318-325, Barcelona, Spain, October 10-14, 2004.

S. A. Abdallah and M. D. Plumbley. Application of Geometric Dependency Analysis to the Separation of Convolved Mixtures. In Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004), Granada, Spain, September 22-24, pp 540-547, 2004.

S. A. Abdallah and M. D. Plumbley: Unsupervised Onset Detection: a Probabilistic Approach using ICA and a Hidden Markov Classifier. In Cambridge Music Processing Colloquium, Cambridge, UK, 2003.

S. A. Abdallah and M. D. Plumbley: Probability as metadata: Event detection in music using ICA as a conditional density model. In Proc 4th Intl. Symp. on Independent Component Analysis and Signal Separation (ICA2003), Nara, Japan, pp 233-238, 2003.

S. A. Abdallah and M. D. Plumbley: Geometric ICA Using Nonlinear Correlation and MDS. In Proc 4th Intl. Symp. on Independent Component Analysis and Signal Separation (ICA2003), Nara, Japan, pp 161-166, 2003.

S. A. Abdallah and M. D. Plumbley: An Independent Component Analysis Approach to Automatic Music Transcription. In Proceedings of the 114th Convention of the Audio Engineering Society, Amsterdam, March 2003.

J. P. Bello, L. Daudet, S. A. Abdallah, C. Duxbury, M. E. Davies and M. B. Sandler: A Tutorial on Onset Detection in Music Signals. To appear in IEEE Transactions on Speech and Audio Processing.

T. Blumensath and M. E. Davies, Unsupervised learning of sparse and shift-invariant decompositions of polyphonic music. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), pp V: 497-500, May 2004.

T. Blumensath and M. E. Davies, On Shift-Invariant Sparse Coding. In Proceedings of the International Conference on Independent Component Analysis (ICA 2004), Granada (Spain), September 2004.

E. Oja and M. D. Plumbley. Blind Separation of Positive Sources by Globally Convergent Gradient Search. Neural Computation 16(9), pp 1811-1825, Sept 2004.

E. Oja and M. D. Plumbley. Blind Separation of Positive Sources using Non-Negative PCA. In Proceedings of the Fourth International Symposium on Independent Component Analysis (ICA2003), pp 11-16, Nara, Japan, April 1-4, 2003.

M. D. Plumbley, S. A. Abdallah, T. Blumensath and M. E. Davies: Sparse Representations of Polyphonic Music. Submitted for publication.

M. D. Plumbley. Geometrical Methods for Non-negative ICA: Manifolds, Lie Groups and Toral Subalgebras. To appear in Neurocomputing.

M. D. Plumbley. Optimization using Fourier Expansion over a Geodesic for Non-Negative ICA. To appear in Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004), Granada, Spain, September 22-24, 2004.

M. D. Plumbley. Lie Group Methods for Optimization with Orthogonality Constraints. To appear in Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation (ICA 2004), Granada, Spain, September 22-24, 2004.

M. D. Plumbley and E. Oja. A 'Non-Negative PCA' Algorithm for Independent Component Analysis. IEEE Transactions on Neural Networks, 15(1), pp 66-76, Jan 2004.

M. D. Plumbley. Algorithms for non-negative independent component analysis. IEEE Transactions on Neural Networks, 14(3), pp 534- 543, May 2003.

M. D. Plumbley. Conditions for non-negative independent component analysis. IEEE Signal Processing Letters, 9(6) , pp177 -180, June 2002.

M. D. Plumbley, S. A. Abdallah, J. P. Bello, M. E. Davies, G. Monti, and M. B. Sandler: Automatic Music Transcription and Audio Source Separation. Cybernetics and Systems, 33(6):603-627, 2002.
[Download preprint: article (pdf: 52k) and figures (pdf: 315k). NB Fig 8 may render very slowly.]

Software

The following software resulting from this project is available for download:

• JSLAB - A Java class library and a bunch of scheme scripts which provide an interactive, interpreted environment for doing mathematical and signal processing experiments.
• QMLAB - Miscellaneous Matlab tools for ICA, sparse coding, multidimensional scaling, etc.

Participants

Dr Mark Plumbley
Prof Mark Sandler
Dr Mike Davies
Dr Samer Abdallah
Mr Thomas Blumensath

Duration

Jan 2002 to June 2004

Sponsor

EPSRC Grant GR/R54620/01: £124,946