COST 292

www.cost292.org

Objectives and benefits

The main objective of this action is to push forward the frontiers of current research on semantic analysis, inference and conceptualisation for high-level annotation and retrieval of digital audiovisual content.

To achieve this goal this action will bring together leading European research teams working on knowledge-assisted semantic analysis, unification, inference and conceptualisation for high-level recognition of digital content. In particular, the proposed integrative research will seek solutions to issues for which current approaches fail giving focused attention to two main aspects: Semantic learning and inference and multimodal analysis.

Semantic Learning and Inference: The goal is to develop automatic and semi-automatic (e.g. using relevance feedback) approaches to detect and recognize semantically meaningful scenes, objects and events present in the content. This process requires the association of low-level and mid-level automatically extracted features with higher-level semantic concepts.

It is envisioned to develop and test algorithms for:

•  Fast uncompressed and compressed domain sound, audio and image feature extraction (based on MP3, MP4, MPEG-2/4)

•  Integration of all available visual information (colour, texture, shape, motion).

•  Efficient means for description of features using hierarchical or multiresolution descriptors.

Multimodal Based Retrieval Mechanisms: The aim is to develop retrieval mechanisms that fully exploit multimodal features and inferred concepts . Given that each concept will rely on many different features and associations, drawn from different input media sources, it is a significant research challenge to investigate how to combine these features and concepts to best answer a user’s query.

As in any retrieval task, interface design for browsing, search and retrieval from large repositories of content using low-level features, higher-level semantics and user’s relevance feed-back will be also considered.

Main Measurable and Achievable Outcome

The main measurable outcome of the envisaged research will be a modular software system for semantic driven annotation and retrieval. The actual annotation interface and search engine of this COST action should feature two main functionalities:

•  Automatic audio-visual content annotation using concepts and levels of abstraction humans are familiar with.

•  The ability to build up a knowledge base from past experience through user interaction (relevance feed-back) and adaptive learning.

Scientist in charge

Prof Ebroul Izquierdo