Fusion Strategies for Large-Scale Multi-modal Image Retrieval

D 2017

Fusion Strategies for Large-Scale Multi-modal Image Retrieval

BUDÍKOVÁ, Petra, Michal BATKO and Pavel ZEZULA

Basic information

Original name

Fusion Strategies for Large-Scale Multi-modal Image Retrieval

Authors

BUDÍKOVÁ, Petra (203 Czech Republic, belonging to the institution), Michal BATKO (203 Czech Republic, belonging to the institution) and Pavel ZEZULA (203 Czech Republic, belonging to the institution)

Edition

Berlin, Heidelberg, Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII, p. 146-184, 39 pp. 2017

Publisher

Springer

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

printed version "print"

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/17:00094968

Organization unit

Faculty of Informatics

ISBN

978-3-662-55695-5

ISSN

DOI

http://dx.doi.org/10.1007/978-3-662-55696-2_5

Keywords in English

Multimodal image retrieval; fusion strategies; evaluation

Abstract

V originále

Large-scale data management and retrieval in complex domains such as images, videos, or biometrical data remains one of the most important and challenging information processing tasks. Even after two decades of intensive research, many questions still remain to be answered before working tools become available for everyday use. In this work, we focus on the practical applicability of different multi-modal retrieval techniques. Multi-modal searching, which combines several complementary views on complex data objects, follows the human thinking process and represents a very promising retrieval paradigm. However, a rapid development of modality fusion techniques in several diverse directions and a lack of comparisons between individual approaches have resulted in a confusing situation when the applicability of individual solutions is unclear. Aiming at improving the research community’s comprehension of this topic, we analyze and systematically categorize existing multimodal search techniques, identify their strengths, and describe selected representatives. In the second part of the paper, we focus on the specific problem of large-scale multi-modal image retrieval on the web. We analyze the requirements of such task, implement several applicable fusion methods, and experimentally evaluate their performance in terms of both efficiency and effectiveness. The extensive experiments provide a unique comparison of diverse approaches to modality fusion in equal settings on two large real-world datasets.

Links

GA16-18889S, research and development project

Name: Analytika pro velká nestrukturovaná data (Acronym: Big Data Analytics for Unstructured Data)

Investor: Czech Science Foundation

Citovat

BUDÍKOVÁ, Petra, Michal BATKO and Pavel ZEZULA. Fusion Strategies for Large-Scale Multi-modal Image Retrieval. In Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII. Berlin, Heidelberg: Springer, 2017, p. 146-184. ISBN 978-3-662-55695-5. Available from: https://dx.doi.org/10.1007/978-3-662-55696-2_5.

@inproceedings{1390178,
   author = {Budíková, Petra and Batko, Michal and Zezula, Pavel},
   address = {Berlin, Heidelberg},
   booktitle = {Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII},
   doi = {http://dx.doi.org/10.1007/978-3-662-55696-2_5},
   keywords = {Multimodal image retrieval; fusion strategies; evaluation},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Berlin, Heidelberg},
   isbn = {978-3-662-55695-5},
   pages = {146-184},
   publisher = {Springer},
   title = {Fusion Strategies for Large-Scale Multi-modal Image Retrieval},
   year = {2017}
}

TY  - JOUR
ID  - 1390178
AU  - Budíková, Petra - Batko, Michal - Zezula, Pavel
PY  - 2017
TI  - Fusion Strategies for Large-Scale Multi-modal Image Retrieval
PB  - Springer
CY  - Berlin, Heidelberg
SN  - 9783662556955
KW  - Multimodal image retrieval
KW  - fusion strategies
KW  - evaluation
N2  - Large-scale data management and retrieval in complex domains such as images, videos, or biometrical data remains one of the most important and challenging information processing tasks. Even after two decades of intensive research, many questions still remain to be answered before working tools become available for everyday use. In this work, we focus on the practical applicability of different multi-modal retrieval techniques. Multi-modal searching, which combines several complementary views on complex data objects, follows the human thinking process and represents a very promising retrieval paradigm. However, a rapid development of modality fusion techniques in several diverse directions and a lack of comparisons between individual approaches have resulted in a confusing situation when the applicability of individual solutions is unclear. Aiming at improving the research community’s comprehension of this topic, we analyze and systematically categorize existing multimodal search techniques, identify their strengths, and describe selected representatives. In the second part of the paper, we focus on the specific problem of large-scale multi-modal image retrieval on the web. We analyze the requirements of such task, implement several applicable fusion methods, and experimentally evaluate their performance in terms of both efficiency and effectiveness. The extensive experiments provide a unique comparison of diverse approaches to modality fusion in equal settings on two large real-world datasets.
ER  -

BUDÍKOVÁ, Petra, Michal BATKO and Pavel ZEZULA. Fusion Strategies for Large-Scale Multi-modal Image Retrieval. In \textit{Transactions on Large-Scale Data- and Knowledge-Centered Systems XXXIII}. Berlin, Heidelberg: Springer, 2017, p.~146-184. ISBN~978-3-662-55695-5. Available from: https://dx.doi.org/10.1007/978-3-662-55696-2\_{}5.

Detailed Information on Publication Record