Towards Hybrid Evaluation Methodologies for Large Language
Models in the Legal Domain

D 2024

Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain

SANCHI, Marco a Tereza NOVOTNÁ

Základní údaje

Originální název

Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain

Autoři

SANCHI, Marco a Tereza NOVOTNÁ

Vydání

Amsterdam, Berlin, Washington DC, Frontiers in Artificial Intelligence and Applications, Vol. 395 Legal Knowledge and Information Systems. Proceedings of JURIX 2024. od s. 389-392, 4 s. 2024

Nakladatel

IOS Press BV

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

50501 Law

Stát vydavatele

Nizozemské království

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

Open access sborníku

Označené pro přenos do RIV

Ano

Kód RIV

RIV/00216224:14220/24:00138299

Organizační jednotka

Právnická fakulta

ISBN

978-1-64368-562-5

ISSN

DOI

https://doi.org/10.3233/FAIA241279

EID Scopus

2-s2.0-85217085401

Klíčová slova anglicky

Large Language Models; Thematic Analysis; Performance Evaluation

Štítky

rivok

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 4. 4. 2025 13:29, Mgr. Petra Georgala

Anotace

V originále

This paper analyses automated and human-driven evaluation approaches for Large Language Models (LLMs) performance in the legal domain, stressing the need to combine both into hybrid evaluation frameworks. This conclusion is reinforced by a qualitative case study that uncovers assessment factors considered by lawyers when using LLMs. The diverse nature of these factors, requiring distinct evaluation approaches, underscores the need for adopting a hybrid methodology.

Citovat

SANCHI, Marco a Tereza NOVOTNÁ. Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain. Online. In Jaromir Savelka, Jakub Harasta, Tereza Novotna, Jakub Misek. Frontiers in Artificial Intelligence and Applications, Vol. 395 Legal Knowledge and Information Systems. Proceedings of JURIX 2024. Amsterdam, Berlin, Washington DC: IOS Press BV, 2024, s. 389-392. ISBN 978-1-64368-562-5. Dostupné z: https://doi.org/10.3233/FAIA241279.

@inproceedings{2465215,
   author = {Sanchi, Marco and Novotná, Tereza},
   address = {Amsterdam, Berlin, Washington DC},
   booktitle = {Frontiers in Artificial Intelligence and Applications, Vol. 395 Legal Knowledge and Information Systems. Proceedings of JURIX 2024.},
   doi = {https://doi.org/10.3233/FAIA241279},
   editor = {Jaromir Savelka, Jakub Harasta, Tereza Novotna, Jakub Misek},
   keywords = {Large Language Models; Thematic Analysis; Performance Evaluation},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Amsterdam, Berlin, Washington DC},
   isbn = {978-1-64368-562-5},
   pages = {389-392},
   publisher = {IOS Press BV},
   title = {Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain},
   url = {https://ebooks.iospress.nl/volume/legal-knowledge-and-information-systems-jurix-2024-the-thirty-seventh-annual-conference-brno-czech-republic-11-13-december-2024},
   year = {2024}
}

TY  - CONF
ID  - 2465215
AU  - Sanchi, Marco - Novotná, Tereza
PY  - 2024
TI  - Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain
PB  - IOS Press BV
CY  - Amsterdam, Berlin, Washington DC
SN  - 9781643685625
KW  - Large Language Models
KW  - Thematic Analysis
KW  - Performance Evaluation
UR  - https://ebooks.iospress.nl/volume/legal-knowledge-and-information-systems-jurix-2024-the-thirty-seventh-annual-conference-brno-czech-republic-11-13-december-2024
N2  - This paper analyses automated and human-driven evaluation approaches for Large Language Models (LLMs) performance in the legal domain, stressing the need to combine both into hybrid evaluation frameworks. This conclusion is reinforced by a qualitative case study that uncovers assessment factors considered by lawyers when using LLMs. The diverse nature of these factors, requiring distinct evaluation approaches, underscores the need for adopting a hybrid methodology.
ER  -

SANCHI, Marco a Tereza NOVOTNÁ. Towards Hybrid Evaluation Methodologies for Large Language Models in the Legal Domain. Online. In Jaromir Savelka, Jakub Harasta, Tereza Novotna, Jakub Misek. \textit{Frontiers in Artificial Intelligence and Applications, Vol. 395 Legal Knowledge and Information Systems. Proceedings of JURIX 2024.}. Amsterdam, Berlin, Washington DC: IOS Press BV, 2024, s.~389-392. ISBN~978-1-64368-562-5. Dostupné z: https://doi.org/10.3233/FAIA241279.

Přehled o publikaci