Can we detect ChatGPT-generated texts in Czech and Slovak languages?

ŠIGUT, Petr and Tomáš FOLTÝNEK. Can we detect ChatGPT-generated texts in Czech and Slovak languages? In Aleš Horák, Pavel Rychlý, Adam Rambousek. Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2023. Brno: Tribun EU, 2023, p. 35-43. ISBN 978-80-263-1793-7.

Basic information

• Original name: Can we detect ChatGPT-generated texts in Czech and Slovak languages?
• Authors: ŠIGUT, Petr (703 Slovakia, belonging to the institution) and Tomáš FOLTÝNEK (203 Czech Republic, guarantor, belonging to the institution).
• Edition: Brno, Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2023, p. 35-43, 9 pp. 2023.
• Publisher: Tribun EU

Other information

• Original language: English
• Type of outcome: Proceedings paper
• Field of Study: 10200 1.2 Computer and information sciences
• Country of publisher: Czech Republic
• Confidentiality degree: is not subject to a state or trade secret
• Publication form: printed version "print"
• WWW: URL
• RIV identification code: RIV/00216224:14330/23:00132775
• Organization unit: Faculty of Informatics
• ISBN: 978-80-263-1793-7
• ISSN: 2336-4289
• Keywords in English: ChatGPT; AI-detection; Czech; Slovak
• Tags: Reviewed

Abstract

The wide availability of generative AI exacerbates existing threats to society. It would not be easy even for linguists to tell whether the text we are reading was generated by a Large Language Model (LLM) or written by a human. Researchers have started developing tools that detect AI-generated content. This paper tested how two of these tools, Compilatio and GPT-2 Output Detector, performed with Czech, Slovak and English texts. There was only one tool somewhat capable of detecting AI-generated texts: Compilatio. Other tools were designed to work only with English texts. Hence, we also tested whether automatically translating the Czech and Slovak texts to English before uploading them to the detectors would have given any promising results. Ultimately, we showed that the texts generated by ChatGPT4 were less detectable than the texts generated by ChatGPT3.5.