MANSOURI, Behrooz, Vít NOVOTNÝ, Anurag AGARWAL, Douglas W. OARD and Richard ZANIBBI. Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version). Online. In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, Martin Potthast. Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. Bologna: CEUR-WS, 2022, p. 1-27. ISSN 1613-0073.
Other formats:   BibTeX LaTeX RIS
Basic information
Original name Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)
Authors MANSOURI, Behrooz, Vít NOVOTNÝ (203 Czech Republic, guarantor, belonging to the institution), Anurag AGARWAL, Douglas W. OARD and Richard ZANIBBI.
Edition Bologna, Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, p. 1-27, 27 pp. 2022.
Publisher CEUR-WS
Other information
Original language English
Type of outcome Proceedings paper
Field of Study 10201 Computer sciences, information science, bioinformatics
Country of publisher Italy
Confidentiality degree is not subject to a state or trade secret
Publication form electronic version available online
WWW fulltext PDF
RIV identification code RIV/00216224:14330/22:00127647
Organization unit Faculty of Informatics
ISSN 1613-0073
Keywords in English information retrieval; sparse retrieval; dense retrieval; soft vector space model; math representations; word embeddings; constrained positional weighting; decontextualization; word2vec; transformers
Tags evaluation, information retrieval, math indexing and retrieval, math information retrieval, MathML, Positional embeddings, ranking, SCM, similarity search, soft cosine measure
Tags International impact, Reviewed
Changed by Changed by: RNDr. Pavel Šmerk, Ph.D., učo 3880. Changed: 6/4/2023 09:37.
Abstract
This paper provides an overview of the third and final year of the Answer Retrieval for Questions on Math (ARQMath-3) lab, run as part of CLEF 2022. ARQMath has aimed to introduce test collections for math-aware information retrieval. ARQMath-3 has two main tasks, Answer Retrieval (Task 1) and Formula Search (Task 2), along with a new pilot task Open Domain Question Answering (Task 3). Nine teams participated in ARQMath-3, submitting 33 runs for Task 1, 19 runs for Task 2, and 13 runs for Task 3. Tasks, topics, evaluation protocols, and results for each task are presented in this lab overview.
PrintDisplayed: 2/5/2024 13:08