Obsah obrázku socha, umění Popis byl vytvořen automaticky 22.10.2024 GLCb2028 Artificial Intelligence in Political Science and Security Studies Jan KLEINER jkleiner@mail.muni.cz Science and AI: Building a Toolbox Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Lecture outline •Part 1: How science works and academic ethics à the basis for ethical AI use. •Part 2: Toolbox and practical examples. •Part 3: Literature review. Obsah obrázku socha, umění Popis byl vytvořen automaticky Part 1: Science Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky What is Science? •Common basis for all disciplines (Gauch, 2003): •Evidence-based. •Use of inductive (qualitative) and deductive logic (hypotheses testing). •Probability. •Parsimony. •Principles of science: •Empirically testable. •Replicable. •Objective/non-personal. •Transparent/controlable (by other scientists). •Falsifiable/confirmable (inferences must relate to observed evidence). •Logical Consistency. •Broader context. • • • Obsah obrázku socha, umění Popis byl vytvořen automaticky Important criteria 1. 1. •Validity and reliability – favours quantitative. •What about qualitative? •Mason (1996): rigour, quality and potential for generalizability. •Le Compte and Goetz (1982): external reliability (replicability) and validity (generalizability), internal reliability (among observers) and validity (aligning data and theory). •Guba and Lincoln (1981): Trustworthiness set: credibility (via triangulation), transferability (to different case), dependability (due dilligence? + audit trail), confirmability (on whose side are we? We cannot do impartial research; transparencyt). • Obsah obrázku socha, umění Popis byl vytvořen automaticky Ethics – why? •Keeping the trust in science. •Citations – author contribution acknowledgement. •Effective system of vast knowledge management. •Transparency -> auditability. • •Can you think of any other reasons? Obsah obrázku socha, umění Popis byl vytvořen automaticky Academic hell 1. 1. •Sloppy science – archiving, collecting, storing etc. •Fabrication of data and conclusions. •Wrong use of scientific methods. •Big ego. •Not upholding scientific principles. •Plagiarism. Obsah obrázku socha, umění Popis byl vytvořen automaticky Student-centered principles (UTS, 2023) 1. 1. 1.Students understand the significance of GenAI for society, careers, and studies. 2.Students understand legitimate use of GenAI in their studies. 3.Students are equipped to engage critically and ethically with GenAI. 4.Students experience GenAI’s strengths and limitations as aids to learning. 5.Students are assessed on what they need to know in an AI world. Obsah obrázku socha, umění Popis byl vytvořen automaticky Open Universities Australia (2023) 1. 1. DO: •ask for research guidance before writing an essay •use it when brainstorming •ask questions about study material you don’t understand •use it to proofread your work •cite any AI assistance in your reference list DONT: •ask AI software to write essays for you •blindly trust AI-generated information •do anything that violates your university’s academic integrity policy Obsah obrázku socha, umění Popis byl vytvořen automaticky How to cite ChatGPT? APA •“When prompted with “Is the left-brain right brain divide real or a metaphor?” the ChatGPT-generated text indicated that although the two brain hemispheres are somewhat specialized, “the notation that people can be characterized as ‘left-brained’ or ‘right-brained’ is considered to be an oversimplification and a popular myth” (OpenAI, 2023).“ (McAdoo, 2023) • •Reference •OpenAI. (2023). ChatGPT (Mar 14 version) [Large language model]. https://chat.openai.com/chat Obsah obrázku socha, umění Popis byl vytvořen automaticky Threats to ethics and their mitigation 1. 1. •Plagiarism - especially unintentional - models are fielded on data from scientific papers, among others (Kim, 2024; Khalif et al., 2023) à do not generate (larger) text, use reliable AI tools. •AI often hallucinates (fabricates) citations (Buriak et al., 2024; Zeer et al., 2023) à use reliable AI, check everything thoroughly directly in the cited text. •Misinformation - inaccurate or biased results (Buriak et al., 2024; Kim, 2024), AI suffers from a lot of biases! à triangulation of sources and personal verification of given information. • Obsah obrázku socha, umění Popis byl vytvořen automaticky Practical examples •OK: consultation of methods, theories and overall design of the paper, including occasional proofreading; literature search (Consensus, SciSpace, Research Rabbit, etc.) - but it should be complementary to the traditional search! • •Problematic: whole-text proofreading, small-scale content generation (e.g. statistical results). • •Inacceptable: large-scale content generation. Obsah obrázku socha, umění Popis byl vytvořen automaticky Good practices (Buriak et al., 2023, Khalifa & Albadway, 2024, Khlaif et al., 2023) 1. 1. •Acknowledgements section - acknowledge the use of AI and transparently list the parts of the text affected, possibly including prompts or transcripts in Supplementary files. •Constantly keep in mind - the output from gen AI is only an initial draft and needs significant polishing and rewriting. •Don't use the literal and unedited output of gen AI - remember it cannot generate new, original ideas, on par with ghostwriting. •Don't let AI tools diminish your creativity and thinking à introspection à use them for exactly the opposite purpose! •AI-generated content should never replace critical thinking, human and expert insight and literature review. •All outputs must be thoroughly checked. •Do not input personal, sensitive, or otherwise protected data and text into AI. • • Obsah obrázku socha, umění Popis byl vytvořen automaticky Part 2: Toolbox Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Generative AI and study companions 1. 1. •Claude – LLM like ChatGPT, but higher limit (only in USA and UK). •+ ChatGPT and Bard. •+ ChatGPT cheat sheets. •Revision.ai – Flashcards from pdf notes. •Easy-Peasy AI – GPT-4 model based AI + image gen. •Quillbot – paraphraser, MS Word plugin. •Forefront.ai – GPT-3.5, Claude, input tokens, internet search, file uploads •PromptGPT – web with prompts and jailbreaks. •Prompt-genie – 14 days free trial. •Scholarcy – summary cards of small/medium documents. •MAXQDA – Open AI plugin – text analysis client-based SW. •Penseum – study companion, temporary halt on new accounts. •Perplexity.ai – ChatGPT + Google •Lens.org – Better Google •+ Testing of SCOPUS Copilot and WoS AI Assistant • • • • • Obsah obrázku socha, umění Popis byl vytvořen automaticky Literature review •Elicit – summarizing, comparing studies free (5k credits) -> paid. •Scispace – pdf TLDR, summaries, chat. •Research Rabbit – vizualisation. •Connected Papers – vizualisation. •Open Knowledge Maps – visualisation. •Consensus – outputs based on a research question. •VOSViewer – client-based SW, visualisation, text mining capabilities. •Sourcerly – find sources. •Jenni AI – writing co-pilot using papers. •Litmaps – lit finder and vis. Obsah obrázku socha, umění Popis byl vytvořen automaticky Data visualisation and analysis 1. 1. •Flourish. •Datawrapper. •BioRender – life sciences visualisations. •Bard – „write a Python/R code for data visualisation“ à paste data sample à export code à use the code in R studio •Text analysis (Claude/ChatGPT – Grounded Theory coding) à sensitizing concepts + Leximancer (free trial) à analysis. •Slow adoption of AI for qualitative analyses – see study. •GPT Excel – MS Excel formulas generator and companion. • •There is an AI for that – find Ais using AI. • • Obsah obrázku socha, umění Popis byl vytvořen automaticky Part 3: Literature review Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Literature review – a little bit of theory (Jesson, Matheson and Lacey, 2011) •Systematic (structured) vs. traditional (unstructured) à continuum! • • • • • • • • • • •Cannot be ever complete – esp. with the problem of GRAY LITERATURE. • • Traditional Systematic Random process guided by researcher (bias) Rigorous, structured process driven by scientific principles and methods. Implicitly/explicitly subjective. Objective, balanced, less biased. Lack of scientific method(s). Method is focused, explicit and transparent. Identification of topic, topic recon, support for arguments, introduction. Systematic overview of every relevant piece of knowledge – available. Source: Moher et al., 2009, p. 5-6 References •Buriak, J. M., Akinwande, D., Artzi, N., Brinker, C. J., Burrows, C., Chan, W. C. W., Chen, C., Chen, X., Chhowalla, M., Chi, L., Chueh, W., Crudden, C. M., Di Carlo, D., Glotzer, S. C., Hersam, M. C., Ho, D., Hu, T. Y., Huang, J., Javey, A., Kamat, P. V., Kim, I.-D., Kotov, N. A., Lee, T. R., Lee, Y. H., Li, Y., Liz-Marzán, L. M., Mulvaney, P., Narang, P., Nordlander, P., Oklu, R., Parak, W. J., Rogach, A. L., Salanne, M., Samorì, P., Schaak, R. E., Schanze, K. S., Sekitani, T., Skrabalak, S., Sood, A. K., Voets, I. K., Wang, S., Wang, S., Wee, A. T. S., & Ye, J. (2023). Best practices for using AI when writing scientific manuscripts. ACS Nano, 17(5), 4091-4093. https://doi.org/10.1021/acsnano.3c01544. •Gauch, Hugh G. Jr. (2003). Scientific Method in Practice. Cambridge University Press. •Guba, E. G., and Lincoln, Y. S. 1981. Effective evaluation: Improving the usefulness of evaluation results through responsive and naturalistic approaches. San Francisco, CA: Jossey-Bass. •Jesson, J., Matheson, L. and Lacey, F. (2011). Doing Your Literature Review: Traditional and Systematic Techniques. Londýn: SAGE. •LeCompte, M. D., and Goetz, J. P. 1982. Problems of Reliability and Validity in Ethnographic Research. Review of Educational Research, 52, 31-60. https://doi.org/10.3102/00346543052001031 •Khalifa, M., & Albadawy, M. (2024). Using artificial intelligence in academic writing and research: An essential productivity tool. Computer Methods and Programs in Biomedicine Update, 5, 100145. https://doi.org/10.1016/j.cmpbup.2024.100145. •Khlaif, Z. N., Mousa, A., Hattab, M. K., Itmazi, J., Hassan, A. A., & Sanmugam, M. (2023). The potential and concerns of using AI in scientific research: ChatGPT performance evaluation. JMIR Medical Education, 9, e47049. https://doi.org/10.2196/47049. •Kim, S.-J. (2024). Research ethics and issues regarding the use of ChatGPT-like artificial intelligence platforms by authors and reviewers: A narrative review. Science Editing, 11(2), 96-106. https://doi.org/10.6087/kcse.343. •Mason, J. (1996). Qualitative researching. Sage Publications, Inc. •McAdoo, T. (2023). APA Style. Available from: https://apastyle.apa.org/blog/how-to-cite-chatgpt. •Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097 •Open Universities Australia. (2023). How you should—and shouldn’t—use ChatGPT as a student. Available from: https://www.open.edu.au/advice/insights/ethical-way-to-use-chatgpt-as-a-student •UTS. (2023). Five principles for the effective ethical use of generative AI. Available from: https://lx.uts.edu.au/collections/artificial-intelligence-in-learning-and-teaching/resources/five-p rinciples-for-effective-ethical-use-generative-ai/ Obsah obrázku socha, umění Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Questions? Jan KLEINER jkleiner@mail.muni.cz Thank you for your attention! Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky