Research Seminars

Sinusoidal Sound Modeling -- Advances and Applications to Time Stretching and Sound Classification

Sylvain Marchand, LaBRI Université Bordeaux

Wednesday 1 March 2006, 4:00pm, Room 105

Abstract

Spectral models attempt to parameterize sound at the basilar membrane of the ear. Thus, sound representations and transformations in these models should be closely linked to the perception. Among those models, sinusoidal modeling deals with partials that are pseudo-sinusoidal tracks for which frequencies and amplitudes continuously evolve slowly with time. This is a generalization of additive (modal) synthesis, and is also related to the physical structure of the sounds. Sinusoidal modeling is extremely useful for many applications such as musical sound transformation (time stretching, pitch shifting, etc.), coding (compression), watermarking, and also classification.

There are several kinds of sinusoidal models. For example, stationary models consider that the sound parameters can be regarded as constant within some short time window. On the contrary, non-stationary models allow these parameter to vary even within this window, and thus handle transients more efficiently. Although basic sinusoidal models are not well-suited for noisy sounds, extension have been proposed recently to handle this kind of sounds.

One major research direction with sinusoidal modeling is the modeling of the parameters of the partials themselves. By re-analyzing the evolutions of the model parameters, we obtain (order-2) parameters of a hierarchical model well-suited for time stretching while preserving musical modulations such as vibrato and tremolo. Moreover, the reanalysis of the spectral parameters turns out to be extremely useful for difficult problems such as lossless compression or source separation for example.

Sinusoidal models require precise analysis and synthesis methods.

The analysis is often consisting of a short-term spectral analysis (extracting spectral peaks) followed by a partial-tracking algorithm used to form partials from these peaks.

Regarding the short-term analysis method, we proposed a new method that extends the classic short-time Fourier analysis by also considering the derivatives of the sound signal, and which can work with very short analysis windows. We have also investigated analysis in the context of non-stationary models.

Partial tracking is certainly more problematic. Too few algorithms were proposed, and we still need evaluation procedures in order to compare them. We proposed a new partial-tracking algorithm using linear prediction to follow the evolutions of the partials trajectories in time. We enhanced this algorithm by considering the high-frequency content of these trajectories in order to chose the best trajectory and also detect note onsets/offsets.

Finally, a very efficient synthesis algorithm (based on a recursive description of the sine function) can reproduce sound in real time from the model parameters. This algorithm allows an extremely fine control of the partials of the sounds while avoiding signal discontinuities as well as numerical imprecision, and with a nearly optimal number of operations per partial. We also consider psychoacoustic phenomena such as masking in order to reduce on-the-fly the number of partials to be synthesized.

We are also developing the InSpect and ReSpect free software programs for the purposes of sound analysis, transformation, and synthesis.

This research is carried out in the context of the SCRIME project which is funded by the DMDTS of the French Culture Ministry, the Aquitaine Regional Council, the General Council of the Gironde Department and IDDAC of the Gironde Department. The SCRIME project is the result of a cooperation convention between the Conservatoire National de Région of Bordeaux, ENSEIRB (school of electronic and computer scientist engineers) and the University of Bordeaux 1. It is made of composers of electro-acoustic music and scientific researchers. It is managed by the LaBRI (laboratory of research in computer science of Bordeaux). Its main goals are research and creation, diffusion, and pedagogy.