D 2022

Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

MANSOURI, Behrooz, Vít NOVOTNÝ, Anurag AGARWAL, Douglas W. OARD, Richard ZANIBBI et. al.

Basic information

Original name

Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

Authors

MANSOURI, Behrooz, Vít NOVOTNÝ (203 Czech Republic, guarantor, belonging to the institution), Anurag AGARWAL, Douglas W. OARD and Richard ZANIBBI

Edition

Bologna, Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, p. 1-27, 27 pp. 2022

Publisher

CEUR-WS

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Italy

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

References:

RIV identification code

RIV/00216224:14330/22:00127647

Organization unit

Faculty of Informatics

ISSN

Keywords in English

information retrieval; sparse retrieval; dense retrieval; soft vector space model; math representations; word embeddings; constrained positional weighting; decontextualization; word2vec; transformers

Tags

Tags

International impact, Reviewed
Změněno: 6/4/2023 09:37, RNDr. Pavel Šmerk, Ph.D.

Abstract

V originále

This paper provides an overview of the third and final year of the Answer Retrieval for Questions on Math (ARQMath-3) lab, run as part of CLEF 2022. ARQMath has aimed to introduce test collections for math-aware information retrieval. ARQMath-3 has two main tasks, Answer Retrieval (Task 1) and Formula Search (Task 2), along with a new pilot task Open Domain Question Answering (Task 3). Nine teams participated in ARQMath-3, submitting 33 runs for Task 1, 19 runs for Task 2, and 13 runs for Task 3. Tasks, topics, evaluation protocols, and results for each task are presented in this lab overview.