Overview of ARQMath-3 (2022): Third CLEF Lab on Answer
Retrieval for Questions on Math (Working Notes Version)

D 2022

Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

MANSOURI, Behrooz, Vít NOVOTNÝ, Anurag AGARWAL, Douglas W. OARD, Richard ZANIBBI et. al.

Základní údaje

Originální název

Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

Autoři

MANSOURI, Behrooz, Vít NOVOTNÝ (203 Česká republika, garant, domácí), Anurag AGARWAL, Douglas W. OARD a Richard ZANIBBI

Vydání

Bologna, Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, od s. 1-27, 27 s. 2022

Nakladatel

CEUR-WS

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Itálie

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

fulltext PDF

Kód RIV

RIV/00216224:14330/22:00127647

Organizační jednotka

Fakulta informatiky

ISSN

Klíčová slova anglicky

information retrieval; sparse retrieval; dense retrieval; soft vector space model; math representations; word embeddings; constrained positional weighting; decontextualization; word2vec; transformers

Štítky

evaluation, information retrieval, math indexing and retrieval, math information retrieval, MathML, Positional embeddings, ranking, SCM, similarity search, soft cosine measure

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 6. 4. 2023 09:37, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

This paper provides an overview of the third and final year of the Answer Retrieval for Questions on Math (ARQMath-3) lab, run as part of CLEF 2022. ARQMath has aimed to introduce test collections for math-aware information retrieval. ARQMath-3 has two main tasks, Answer Retrieval (Task 1) and Formula Search (Task 2), along with a new pilot task Open Domain Question Answering (Task 3). Nine teams participated in ARQMath-3, submitting 33 runs for Task 1, 19 runs for Task 2, and 13 runs for Task 3. Tasks, topics, evaluation protocols, and results for each task are presented in this lab overview.

Citovat

MANSOURI, Behrooz, Vít NOVOTNÝ, Anurag AGARWAL, Douglas W. OARD a Richard ZANIBBI. Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version). Online. In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, Martin Potthast. Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum. Bologna: CEUR-WS, 2022, s. 1-27. ISSN 1613-0073.

@inproceedings{2243240,
   author = {Mansouri, Behrooz and Novotný, Vít and Agarwal, Anurag and W. Oard, Douglas and Zanibbi, Richard},
   address = {Bologna},
   booktitle = {Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum},
   editor = {Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, Martin Potthast},
   keywords = {information retrieval; sparse retrieval; dense retrieval; soft vector space model; math representations; word embeddings; constrained positional weighting; decontextualization; word2vec; transformers},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Bologna},
   pages = {1-27},
   publisher = {CEUR-WS},
   title = {Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)},
   url = {http://ceur-ws.org/Vol-3180/paper-01.pdf},
   year = {2022}
}

TY  - JOUR
ID  - 2243240
AU  - Mansouri, Behrooz - Novotný, Vít - Agarwal, Anurag - W. Oard, Douglas - Zanibbi, Richard
PY  - 2022
TI  - Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)
PB  - CEUR-WS
CY  - Bologna
KW  - information retrieval
KW  - sparse retrieval
KW  - dense retrieval
KW  - soft vector space model
KW  - math representations
KW  - word embeddings
KW  - constrained positional weighting
KW  - decontextualization
KW  - word2vec
KW  - transformers
UR  - http://ceur-ws.org/Vol-3180/paper-01.pdf
N2  - This paper provides an overview of the third and final year of the Answer Retrieval for Questions on Math (ARQMath-3) lab, run as part of CLEF 2022. ARQMath has aimed to introduce test collections for math-aware information retrieval. ARQMath-3 has two main tasks, Answer Retrieval (Task 1) and Formula Search (Task 2), along with a new pilot task Open Domain Question Answering (Task 3). Nine teams participated in ARQMath-3, submitting 33 runs for Task 1, 19 runs for Task 2, and 13 runs for Task 3. Tasks, topics, evaluation protocols, and results for each task are presented in this lab overview.
ER  -

MANSOURI, Behrooz, Vít NOVOTNÝ, Anurag AGARWAL, Douglas W. OARD a Richard ZANIBBI. Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version). Online. In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, Martin Potthast. \textit{Proceedings of the Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum}. Bologna: CEUR-WS, 2022, s.~1-27. ISSN~1613-0073.

Podrobný výpis o publikaci