Research Seminars

Acoustic lexemes for creative organization of audio

Michael A. Casey, Goldsmiths College, University of London

Thur 30 June 2005

Abstract

We propose a method for automatic fine-scale audio description that draws inspiration from ontological sound description methods such as Shaeffer's Objets Sonores and Smalley's Spectromorphology. Our goal is complete automation of audio description at the level of sound objects for indexing and retrieving sound segments within audio documents.

To automatically segment audio documents into acoustic lexemes, we employ a hidden Markov model with an entropic prior to bias the model towards sparcity and determinism. We propose that the symbol stream of cluster labels, generated by the Viterbi algorithm, constitutes a detailed description of audio as a sequence of spectral archetypes. The ASCII base-64 encoding scheme maps cluster indices to one-character symbols which are then segmented into 8-gram sequences for indexing in a relational database. This representation admits very efficient one-dimensional indexing methods, such as the B-Tree, for segment-level retrieval from very large audio document collections.

We give details of how the system will scale to audio mosaicing with a 1PByte internet audio archive-- employing an index over all acoustic lexeme strings. Our current corpus of 128 full Electroacoustic and 20th-century orchestral works, 1.6M segments, was chosen for its non-conformance to traditional musical materials and structures thereby illustrating immediate application to new music creativity.

Biography:

Michael Casey is senior lecturer in Computing at Goldsmiths College's Centre for Cognition, Computation and Culture. Between 1992 and 2002 Michael was a research fellow at the MIT Media Laboratory and then full-time resaerch scientist at Mitsubishi Electric Research Labs (MERL) where he conducted research in ICA-based spectral subspace separation and machine learning methods for general sound recognition. He is named as inventor on five patents for Mitsubishi Electric and was an editor of the MPEG-7 international standard. As a composer, Michael's aesthetic is based on formal organization of audio using spectromorphology as a structuring principle. He has received international prizes from Bourges and Newcomp for his musical works in the Electroacoustic genre.