Emergence of semantics from multimedia databases

Erkki Oja

Computer Science and Engineering

Helsinki University of Technology, Helsinki, Finland

Abstract
    Digital image and video libraries are becoming more common and widely used, as visual information is produced at a rapidly growing rate. Creating and storing these collections is easy, but there is an increasing need for effective ways to manage and process such information. Content-based image and video retrieval systems are gaining popularity for online access to unannotated image and video databases. For computational reasons, fairly low-level visual and auditory features have to be used for measuring object similarity, which is the basis for clustering and search. A challenging problem is the inherently weak connection between such low-level features and the actual high-level semantic concepts that humans associate with pictures and videos, including sounds and speech. Methods for bridging this semantic gap are urgently needed.
    The author's group has previously introduced the PicSOM system for content-based retrieval of images and multimedia documents. It is based on Self-Organizing Maps (SOM) for features. Each image or multimedia document is mapped onto the feature maps, and the weighting between different maps in content-based search is guided by relevance feedback from the user. When the PicSOM system is equipped with automatic image and video segmentation, and some metadata such as keywords are available, the keyword annotations given on image level can be focused on image frames and segments. In the lack of keywords, such metadata can be automatically produced from recorded online use of the system. This high-level abstracted information can be used to improve the accuracy of retrieval, as well as to categorize the objects in the database with semantic concepts.

Biosketch:
    Erkki Oja received the Dr.Sc. degree from Helsinki University of Technology in 1977. He is Director of the Adaptive Informatics Research Centre and Professor of Computer Science at the Laboratory of
Computer and Information Science, Helsinki University of Technology, Finland. He has been research associate at Brown University, Providence, RI, and visiting professor at the Tokyo Institute of Technology, Japan. He is the author or coauthor of more than 280 articles and book chapters on pattern recognition, computer vision, and neural computing, and three books: "Subspace Methods of Pattern Recognition" (New York: Research Studies Press and Wiley, 1983), which has been translated into Chinese and Japanese; "Kohonen Maps" (Amsterdam, The Netherlands: Elsevier, 1999), and "Independent Component Analysis" (New York: Wiley, 2001), translated into Japanese. His research interests are in the study of principal component and independent component analysis, self-organization, statistical pattern recognition, and applying artificial neural networks to computer vision and signal processing.
    Prof. Oja is a member of the Finnish Academy of Sciences, Founding Fellow of the International Association of Pattern Recognition (IAPR), Past President of the European Neural Network Society (ENNS), and Governing Board member of the International Neural Network Society (INNS). He is also a member of the editorial boards of several journals and has been in the program committees of several recent conferences including the International Conference on Artificial Neural Networks (ICANN), International Joint Conference on Neural Networks (IJCNN), and International Conference on Neural Information Processing (ICONIP). Prof. Oja is the recipient of the 2004 IAPR Pierre Devijver Award and the 2006 IEEE Computational Intelligence Society Neural Networks Pioneer Award.