It Cannot Be Right If It Was Written by AI: On Lawyers'
Preferences of Documents Perceived as Authored by an LLM vs a
Human

J 2024

It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human

HARAŠTA, Jakub, Tereza NOVOTNÁ a Jaromír ŠAVELKA

Základní údaje

Originální název

It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human

Autoři

HARAŠTA, Jakub, Tereza NOVOTNÁ a Jaromír ŠAVELKA

Vydání

Artificial Intelligence and Law, Springer Netherlands, 2024, 0924-8463

Další údaje

Jazyk

angličtina

Typ výsledku

Článek v odborném periodiku

Obor

50501 Law

Stát vydavatele

Německo

Utajení

není předmětem státního či obchodního tajemství

Odkazy

Text na webu časopisu

Impakt faktor

Impact factor: 4.100 v roce 2022

Organizační jednotka

Právnická fakulta

DOI

http://dx.doi.org/10.1007/s10506-024-09422-w

UT WoS

001368851900001

Klíčová slova anglicky

Generative AI; GenAI; Large Language Model; LLM; Automatic Text Generation; Legal Document; Perception of AI-generated Content

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 19. 12. 2024 06:14, JUDr. Mgr. Jakub Harašta, Ph.D.

Anotace

V originále

Large Language Models (LLMs) enable a future in which certain types of legal documents may be generated automatically. This has a great potential to streamline legal processes, lower the cost of legal services, and dramatically increase access to justice. While many researchers focus on proposing and evaluating LLM-based applications supporting tasks in the legal domain, there is a notable lack of investigations into how legal professionals perceive content if they believe an LLM has generated it. Yet, this is a critical point as over-reliance or unfounded scepticism may influence whether such documents bring about appropriate legal consequences. This study is the necessary analysis of the ongoing transition towards mature generative AI systems. Specifically, we examined whether the perception of legal documents' by lawyers and law students (n=75) varies based on their assumed origin (human-crafted vs AI-generated). The participants evaluated the documents, focusing on their correctness and language quality. Our analysis revealed a clear preference for documents perceived as crafted by a human over those believed to be generated by AI. At the same time, most participants expect the future in which documents will be generated automatically. These findings could be leveraged by legal practitioners, policymakers, and legislators to implement and adopt legal document generation technology responsibly and to fuel the necessary discussions on how legal processes should be updated to reflect recent technological developments.

Návaznosti

MUNI/G/1142/2022, interní kód MU

Název: Forensic Support for Building Trust in Smart Software Ecosystems

Investor: Masarykova univerzita, Forensic Support for Building Trust in Smart Software Ecosystems, INTERDISCIPLINARY - Mezioborové výzkumné projekty

Citovat

HARAŠTA, Jakub, Tereza NOVOTNÁ a Jaromír ŠAVELKA. It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human. Artificial Intelligence and Law. Springer Netherlands, 2024. ISSN 0924-8463. Dostupné z: https://dx.doi.org/10.1007/s10506-024-09422-w.

@article{2447437,
   author = {Harašta, Jakub and Novotná, Tereza and Šavelka, Jaromír},
   doi = {http://dx.doi.org/10.1007/s10506-024-09422-w},
   keywords = {Generative AI; GenAI; Large Language Model; LLM; Automatic Text Generation; Legal Document; Perception of AI-generated Content},
   language = {eng},
   issn = {0924-8463},
   journal = {Artificial Intelligence and Law},
   title = {It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human},
   url = {https://link.springer.com/article/10.1007/s10506-024-09422-w},
   year = {2024}
}

TY  - JOUR
ID  - 2447437
AU  - Harašta, Jakub - Novotná, Tereza - Šavelka, Jaromír
PY  - 2024
TI  - It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human
JF  - Artificial Intelligence and Law
PB  - Springer Netherlands
SN  - 09248463
KW  - Generative AI
KW  - GenAI
KW  - Large Language Model
KW  - LLM
KW  - Automatic Text Generation
KW  - Legal Document
KW  - Perception of AI-generated Content
UR  - https://link.springer.com/article/10.1007/s10506-024-09422-w
N2  - Large Language Models (LLMs) enable a future in which certain types of legal documents may be generated automatically. This has a great potential to streamline legal processes, lower the cost of legal services, and dramatically increase access to justice. While many researchers focus on proposing and evaluating LLM-based applications supporting tasks in the legal domain, there is a notable lack of investigations into how legal professionals perceive content if they believe an LLM has generated it. Yet, this is a critical point as over-reliance or unfounded scepticism may influence whether such documents bring about appropriate legal consequences. This study is the necessary analysis of the ongoing transition towards mature generative AI systems. Specifically, we examined whether the perception of legal documents' by lawyers and law students (n=75) varies based on their assumed origin (human-crafted vs AI-generated). The participants evaluated the documents, focusing on their correctness and language quality. Our analysis revealed a clear preference for documents perceived as crafted by a human over those believed to be generated by AI. At the same time, most participants expect the future in which documents will be generated automatically. These findings could be leveraged by legal practitioners, policymakers, and legislators to implement and adopt legal document generation technology responsibly and to fuel the necessary discussions on how legal processes should be updated to reflect recent technological developments.
ER  -

HARAŠTA, Jakub, Tereza NOVOTNÁ a Jaromír ŠAVELKA. It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human. \textit{Artificial Intelligence and Law}. Springer Netherlands, 2024. ISSN~0924-8463. Dostupné z: https://dx.doi.org/10.1007/s10506-024-09422-w.

Podrobný výpis o publikaci