Research Seminars

Adaptive Learning in a World of Projections

Sergios Theodoridis
Dept. of Informatics and Telecommunications, University of Athens, Greece.
(Distinguished Lecturer, IEEE Signal Processing Society)

Tuesday 27 January 2009, 14:45, Room 105

Abstract

The task of parameter/function estimation has been at the center of scientific attention for a long time and it comes under different names such as filtering, prediction, beamforming, curve fitting, classification, regression. Conventionally, the task has been treated as an optimization task of an appropriately adopted loss function. However, in most of the cases, the choice of the loss function is mainly dictated by its mathematically tractability and not on a physical reasoning related to the specific problem at hand. The task is further complicated when a-priori information, in the form of constraints, becomes available. The presence of constraints in estimation tasks is recently gaining in importance, due to the revival of interest in robust learning schemes.

In this talk, the estimation task is treated in the context of set theoretic estimation arguments. Instead of a single optimal point we are searching for a set of solutions that are in agreement with the available information, which is provided to us in the form of a set of training points and a set of constraints.

The goal of this talk is to present a general tool for parameter/function estimation, both for classification as well as regression tasks, in a time adaptive setting in (infinite dimensional) Reproducing Kernel Hilbert spaces (RKHS). The general framework is that of convex set theory via the powerful and elegant tool of projections.

The structure of this talk evolves along the following directions:

1. It presents in simple geometric arguments the basic principles behind the convex set theoretic approach via projections in the generalized online setting. In contrast to the classical POCS theory, in our generalized methodology the number of convex sets changes in each algorithmic step, as time evolves and new data samples are received. This generalization is necessary for adaptive estimation.

2. It demonstrates the methodology for two case studies of particular interest in the adaptive learning community:

 A generalized kernel-APA (Affine Projection Algorithm) scheme, which is derived irrespective of the differentiability or not of the respective cost function, in the context of the classification task.
 A constrained robust beamforming algorithm, performed in RKHS, as an example of an adaptive constrained estimation, using robust statistics non- differentiable costs.

The focus behind both of these examples will be to demonstrate that the main effort behind our technique consists of shaping the set of constraints and the set of cost function bounds, associated with the training data, in a form of convex sets. In the sequel, linear projections, in an analytic form, can readily be mobilized from a “tool box”, that has been constructed to cover most of the commonly used cost functions. Each algorithmic step consists of a sequence of projections, of linear complexity with respect to the number of unknown parameters. Our theory proves that the algorithm converges to the intersection of all (with a possible exception of a finite number of) the previous convex sets, where the required solution lies.

The work has been carried out in cooperation with Kostas Slavakis and Isao Yamada.

Biography

Sergios Theodoridis is currently Professor of Signal Processing and Communications in the Department of Informatics and Telecommunications of the University of Athens. His research interests lie in the areas of Adaptive Algorithms and Communications, Machine Learning and Pattern Recognition, Signal Processing for Audio Processing and Retrieval. He is the co-editor of the book “Efficient Algorithms for Signal Processing and System Identification”, Prentice Hall 1993, the co-author of the book “Pattern Recognition”, Academic Press, 4th Ed. 2008, and the co-author of three books in Greek, two of them for the Greek Open University.

He has served as President of EURASIP and he is currently a member of the Board of Governors for the IEEE CAS Society. He is the co-author of four papers that have received best paper awards, including the IEEE Computational Intelligence Society Transactions on Neural Networks Outstanding Paper Award. He currently serves as Distinguished Lecturer of the IEEE Signal Processing Society.

He is a member of the Greek National Council for Research and Technology and Chairman of the SP advisory committee for the Edinburgh Research Partnership (ERP). He has served as vice chairman of the Greek Pedagogical Institute and he was for four years member of the Board of Directors of COSMOTE (the Greek mobile phone operating company). He is Fellow of IET and Fellow of IEEE.