Obsah obrázku socha, umění Popis byl vytvořen automaticky 29. 10. 2024 GLCb2028 Artificial Intelligence in Political Science and Security Studies Jan KLEINER jkleiner@mail.muni.cz Intelligence Studies Toolbox: Predictions, Simulations, and Wargames Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky PART I: Intelligence Studies Toolbox: Predictions, Simulations Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Presentation outline • •Intelligence studies introduction. •Particular methods (Center for the Study of Intelligence (U.S.). , 2009): •Diagnostic techniques. •Contrarian techniques. •Imaginative thinking techniques. •Wargames (Part II). •Methods: AI, and Wargames as source of data (Part III). Aim •To gain knowledge of methods used for forecasting – goes hand in hand with simulations. •AI meets forecasting theory and methods. •Caveat: intelligence studies methods are not generally accepted as mainstream social science methods! Obsah obrázku socha, umění Popis byl vytvořen automaticky Key questions • •How to forecast using relatively rigorous methods? • •Can forecasting function as a rigorous social-science methodology? • •How to generate complex simulations that yield valid and reliable data? • •How to employ AI in the whole process? • •Can simulations and wargames be utilized as a rigorous social-science method of data collection? • •Can we center the data collection around AI? • Obsah obrázku socha, umění Popis byl vytvořen automaticky Literature • •Center for the Study of Intelligence (U.S.). (2009). A tradecraft primer: Structured analytic techniques for improving intelligence analysis. U.S. Central Intelligence Agency, Center for the Study of Intelligence. • •Knack, A. & Powell, R. (2023). Artificial Intelligence in Wargaming: An evidence-based assessment of AI applications. The Alan Turing Institute. •A research report, 58 pages, just peruse it quickly to get notion of how the research report looks like and what are its takeaway points. Obsah obrázku socha, umění Popis byl vytvořen automaticky Intelligence studies I (Johnson, 2006) • • •Secretive nature – mostly studied from the outside (insiders need security clearance and are not allowed to publish). • •Interdisciplinary academic field devoted to analyzing intelligence activities, agencies, and processes. • •It examines issues like intelligence collection, analysis, counterintelligence, covert action, and accountability. • •Key topics include the structure/organization of intelligence agencies, the intelligence cycle, and intelligence oversight. • Obsah obrázku socha, umění Popis byl vytvořen automaticky Intelligence studies II (Johnson, 2006) • •Forecasting techniques predict future events and trends. This can involve statistical models, simulations, and expert analysis. • •E.g., forecasts on geopolitical developments, conflicts, weapons proliferation, terrorism threats, and cybersecurity. • •Simulations allow intelligence analysts to model complex situations under different scenarios. This helps test assumptions and identify key variables/uncertainties. • •An "anticipatory" vs. reactive approach. • •However, forecasting faces challenges like cognitive biases, uncertainty, and the difficulty of modelling human behavior. Obsah obrázku socha, umění Popis byl vytvořen automaticky Claude (2023): • • •Business - Forecasting sales, market trends, new product adoption, competitor moves. Competitive intelligence. •Economics - Forecasting economic growth, inflation, unemployment, recessions. Monitoring systemic risks. •Science/Technology - Anticipating new innovations and diffusion patterns. Technology forecasting. •Public Health - Disease outbreak modeling and surveillance. Tracking population health patterns. •Climate/Environment - Projecting impacts of climate change. Modeling environmental risks/catastrophes. •Politics - Election forecasting models. Predicting political instability and regime changes. •Humanitarian - Early warning models for famine, natural disasters, refugee flows. •Crime - Statistical crime prediction models. Pattern analysis of criminal networks. Obsah obrázku socha, umění Popis byl vytvořen automaticky Diagnostic techniques: Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky •Key Assumptions Check. •Quality of Information Check. •Indicators of Signposts of Change. •Analysis of Competing Hypotheses (ACH). Obsah obrázku socha, umění Popis byl vytvořen automaticky Analysis of Competing Hypotheses (ACH): • •Alternative explanations (hypotheses) and evidence that will disconfirm rather than support hypotheses. • •Absorbs large ammount of data. • •Mitigates deception and denial, first impression and confirmation biases. • •Prevents picking on first satisfactory solution and steers analysts to go through all options. • •All the information and argumentation must be evaluated and given equal treatment. Obsah obrázku socha, umění Popis byl vytvořen automaticky ACH steps I: • • 1.Brainstorm to identify all possible hypotheses. 2.List all significant evidence relevant to all the hypotheses. 3.Prepare a matrix with hypotheses. 4.Determine whether each piece of evidence is consistent, inconsistent, or not applicable to each hypothesis. 5.Refine the matrix and reconsider (add) the hypotheses (focus on disproving hypotheses rather than retaining). 6.Analyze how sensitive the ACH results are to a few critical items of evidence; should those pieces prove to be wrong, misleading, or subject to deception, how would it impact an explanation’s validity? Obsah obrázku socha, umění Popis byl vytvořen automaticky ACH steps II: • • Ask what evidence is not being seen but would be expected for a given hypothesis to be true. Is denial and deception a possibility? • Establish the relative likelihood for the hypotheses and report all the conclusions. Obsah obrázku socha, umění Popis byl vytvořen automaticky Contrarian techniques: Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky •Devil’s Advocacy •Team A/Team B •High-Impact/Low-Probability Analysis •“What If?” Analysis Obsah obrázku socha, umění Popis byl vytvořen automaticky Devil´s Advocacy • • •„Challenging a single, strongly held view or consensus by building the best possible case for an alternative explanation.“ • • •Identifying faulty logic. •Challenging key assumptions – can make them stronger through discussion or can discard them. •Identification of alternative explanations. • •World War Z. • • Obsah obrázku socha, umění Popis byl vytvořen automaticky Devil´s Advocacy Steps: • • Outline the mainline judgment and key assumptions and characterize the evidence supporting that current analytic view. • Select one or more assumptions—stated or not—that appear the most susceptible to challenge. • Review the information used to determine whether any is of questionable validity, whether deception is possibly indicated, or whether major gaps exist. • Highlight the evidence that could support an alternative hypothesis or contradicts the current thinking. • Present the findings that demonstrate there are flawed assumptions, poor quality evidence, or possible deception at work. Obsah obrázku socha, umění Popis byl vytvořen automaticky A/B Teaming • • •„Use of separate analytic teams that contrast two (or more) strongly held views or competing hypotheses.“ • •Methods: Analysis and debate phase – contrarian oppinions and discussion (moderated or not). • • Obsah obrázku socha, umění Popis byl vytvořen automaticky High Impact/Low Probability • „Highlights a seemingly unlikely event that would have major policy consequences if it happened.“ •Allows for capturing unlikely scenario with factors at play that can have devastating impact. •Preparing for a black swan. • „Assumes that an event has occurred with potential (negative or positive) impact and explains how it might come about.“ - What were its predictors? What if analysis Obsah obrázku socha, umění Popis byl vytvořen automaticky HILP method: • • •Define the high-impact outcome first (justification). •Devise one or more plausible explanations for or “pathways” to the low probability outcome. •Insert possible triggers or changes in momentum if appropriate (e.g., natural disasters). •Brainstorm plausible but unpredictable triggers of sudden change. •Identify for each pathway a set of indicators or “observables” that would help you anticipate. •Identify factors that would prevent such scenario. Obsah obrázku socha, umění Popis byl vytvořen automaticky Imaginative techniques: Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky •Brainstorming •Outside-in thinking •Red Team Analysis •Alternative Futures Analysis Obsah obrázku socha, umění Popis byl vytvořen automaticky Red team analysis • „Models the behavior of an individual or group by trying to replicate how an adversary would think about an issue.“ •Mitigates the mirror-image bias (methods puts analysts into the shoes of the adversary). The method: •Put themselves in the adversary’s circumstances. •Develop a set of “first-person” questions that the adversary would ask, such as: “How would I perceive incoming information; what would be my personal concerns; or to whom would I look for an opinion?” •Draft a set of policy papers or other simulated documents in which the adversary makes specifc decisions, proposes recommendations, or lays out courses of actions. Obsah obrázku socha, umění Popis byl vytvořen automaticky PART II: Wargames Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky AI and Wargames I (Knack and Powell, 2023) • • •Red Teaming in general (political/security/other simulations, table-tops -> identification of gaps in a strategy, SWOT analyses, policy analyses etc.). •Narrow (safe) usage: Repetitive tasks within sims and wargames (background info creation, automatic translation/transcription, textual data analysis, visuals etc.). •High-risk usage: Red team, game manager etc. •Low cost/questionable reliability. •Better on tactical/operational level than on the strategic one. • Obsah obrázku socha, umění Popis byl vytvořen automaticky Wargame theory – introduction I (Appleget et. al, 2020) 1. 1. •Usually a sponsor – sets goals and timeframe. •Sole purpose of the analytic wargame is to collect analytic data to answer sponsor´s (research) questions – data determine wargame´s success à well tought-out data collection plan is needed! •Roadmap = data collection and management plan (DCMP). •Not just for combat/conflict scenarios, but for Analysis of alternatives (AoA) – e.g., M1A2 Abrams and its replacement options. •+ pedagogic, research tool. Obsah obrázku socha, umění Popis byl vytvořen automaticky Wargame theory – introduction II (Appleget et. al, 2020) 1. 1. •Course of action wargaming. •BOGGSAT = "bunch of guys and gals sitting around a table„. •Vs. •Seminar wargames - designed around the DCMP (Decision-Centric Methodology Process) and have a structured approach. •Quantitative/qualitative/hybrid models. •Strong role of probability and chance (dice rolls) + conditioned probability (e.g., missile interception of Iron Dome AA system – informed by statistics). • • • Source: Appleget et al. (2020, p. 73). Nuclear War Simulator Shows What War With Russia Would Look Like Process: 1.Choose the arena of the simulation – wargame/strategic table-top/policy analysis/COA etc. •Why? – justification, purpose -> litreview 1. 2.Choose the specifics of the simulation framework – actors, setting, dividing into teams, roles etc. 3. 3.Proceed while adhering to the literature. Obsah obrázku socha, umění Popis byl vytvořen automaticky AI and Wargame II (Appleget et al., 2020) • • •Will it be an analytic wargame? If so, how will we collect data? à Data Collection and Management Plan (DCMP). •BOGGSAT vs. seminar wargame (designed around the DCMP)? Do not choose only because it is easy! •Who will be the fascilitator? •Establish a wargame plan – every task that needs to be accomplished for preparation, execution, analysis, and reporting. •Incorporate AI tools into the wargame – adhere to literature! • • • Obsah obrázku socha, umění Popis byl vytvořen automaticky AI and Wargame III 1. 1. •AI can assist in understanding adversaries' perspectives and potential actions (Davis & Bracken, 2022). •AI-directed decisional guidance and computationally informed decision-making can help manage the complexity of modern battlespaces (Kase et al., 2022). •Explainable AI systems can enhance trust and transparency in military training simulations, helping users understand AI decisions and improving overall training effectiveness (Khan et al., 2024). • Obsah obrázku text, snímek obrazovky, Písmo, Paralelní Popis byl vytvořen automaticky DCMP (Appleget et al., 2020, p. 104) Obsah obrázku text, snímek obrazovky, Písmo, Paralelní Popis byl vytvořen automaticky Design (Appleget et al., 2020) •Measurement space (if analytical) – quantitative/qualitative?; conceptualisation and operationalisation; function of the scenario – e.g., TTDs of MBTs). •Scenario – sets scene + narrative for players; must be plausible (we do not want players to „fight the scenario“). •Data – initiate (data we need before the wargame begins), feedback data (needed to keep players playing – injects, responses etc.), analysis data. •Methods, models, and tools. •Players. •Assumptions – all scenarios set in future need assumptions! à time setting needs to be decided as well (past, present, future). •à Practical exercise (p. 128). Obsah obrázku socha, umění Popis byl vytvořen automaticky PART III: Wargames and AI as source of data Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Wargaming as a method 1. 1. •Experimental design: wargames can be structured as controlled experiments to study human behaviour in conflict scenarios (Schechter, Schneider & Shaffer, 2021). •SIGNAL = Strategic Intelligence Game Applied to National and Allied Leadership. •E.g., Nuclear threshold (Reddie & Goldblum 2023): Much higher probability of the low-yield NW than in survey experiments. •Human Decision-Making: wargaming as a rich source of empirical data on human behaviour under stress (Banks, 2023). •Immersive and representative: the nature of wargames (players deeply engaged in realistic scenarios), allows to study behaviours in close to real-life situations (Banks, 2023). •Can generate both quantitative data (e.g. decision outcomes, time taken for decisions) and qualitative data (e.g. player narratives, strategic discussions) (Eremin, 2021). • • Obsah obrázku socha, umění Popis byl vytvořen automaticky AI in social-science methods • • •Generative AI can help generate hypotheses and design experiments by analyzing large and complex datasets (Wang et al., 2023). • •Generative AI can assist in qualitative research by identifying themes from semi-structured interviews. Human evaluators are still needed for nuanced analysis (Spangler et al., 2024). •+ simulations. • Obsah obrázku socha, umění Popis byl vytvořen automaticky Key questions – wrap up • •How to forecast using relatively rigorous methods? • •Can forecasting function as a rigorous social-science methodology? • •How to generate complex simulations that yield valid and reliable data? • •How to employ AI in the whole process? • •Can simulations and wargames be utilized as a rigorous social-science method of data collection? • •Can we center the data collection around AI? • References - Part I •Johnson, L. K. (ed.). (2006). Handbook of intelligence studies. London: Routledge. •Center for the Study of Intelligence (U.S.). (2009). A tradecraft primer: Structured analytic techniques for improving intelligence analysis. U.S. Central Intelligence Agency, Center for the Study of Intelligence. •Claude. (2023, November 6). Give me examples of areas in which forecasting and intelligence studies techniques can be beneficial beyond security. Here are some examples of how forecasting and intelligence studies techniques could provide value beyond security: •Business - Forecasting sales, market trends, new product adoption, competitor moves. Competitive intelligence. •Economics - Forecasting economic growth, inflation, unemployment, recessions. Monitoring systemic risks. •Science/Technology - Anticipating new innovations and diffusion patterns. Technology forecasting. •Public Health - Disease outbreak modeling and surveillance. Tracking population health patterns. •Climate/Environment - Projecting impacts of climate change. Modeling environmental risks/catastrophes. •Politics - Election forecasting models. Predicting political instability and regime changes. •Humanitarian - Early warning models for famine, natural disasters, refugee flows. •Crime - Statistical crime prediction models. Pattern analysis of criminal networks. •Education - Projecting future skills demands. Monitoring student outcomes. •The key is applying intelligence techniques like data gathering, rigorous analysis, modeling, tradecraft to create actionable foresight in new domains. This can improve planning and risk management beyond traditional security realms. Retrived from [https://claude.ai/chat/b5ca606a-860c-150-8d1e-dc933d13a01a] Obsah obrázku socha, umění Popis byl vytvořen automaticky References – Part II •Appleget, J. Burks, R. & Cameron, F. (2020). The Craft of Wargaming – A Detailed Planning Guide for Defense Planners and Analysts. Annapolis: Naval Institute Press. ISBN 9781682473771 •Knack, A. & Powell, R. (2023). Artificial Intelligence in Wargaming An evidence-based assessment of AI applications. The Alan Turing Institute. • Obsah obrázku socha, umění Popis byl vytvořen automaticky References – Part III •Banks, D. E. (2023). The Methodological Machinery of Wargaming: A Path toward Discovering Wargaming’s Epistemological Foundations. International Studies Review, 26(1). https://doi.org/10.1093/isr/viae002 •Davis, P. K., & Bracken, P. (2022). Artificial intelligence for wargaming and modeling. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 154851292110731. https://doi.org/10.1177/15485129211073126 •Eremin, A. (2021). Research Potential of Wargames in International Relations Studies. World Economy and International Relations, 65(12), 90–100. https://doi.org/10.20542/0131-2227-2021-65-12-90-100 •Kase, S. E., Hung, C. P., Krayzman, T., Hare, J. Z., Rinderspacher, B. C., & Su, S. M. (2022). The Future of Collaborative Human-Artificial Intelligence Decision-Making for Mission Planning. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.850628 •Khan, A., Jhanjhi, N. Z., Hamid, D. H. T. B. A. H., & Omar, H. A. H. bin H. (2024). Explainable AI in Military Training Applications (pp. 199–234). https://doi.org/10.4018/978-1-6684-6361-1.ch007 •Reddie, A. W., & Goldblum, B. L. (2023). Evidence of the unthinkable: Experimental wargaming at the nuclear threshold. Journal of Peace Research, 60(5), 760–776. https://doi.org/10.1177/00223433221094734 •Schechter, B., Schneider, J., & Shaffer, R. (2021). Wargaming as a Methodology: The International Crisis Wargame and Experimental Wargaming. Simulation & Gaming, 52(4), 513–526. https://doi.org/10.1177/1046878120987581 •Spangler, S., Beasley, S., & Lingelbach, K. (2024). Analyzing artificial intelligence frameworks: Phenomenologists against machines. Issues In Information Systems. https://doi.org/10.48009/4_iis_2024_126 •Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., Anandkumar, A., Bergen, K., Gomes, C. P., Ho, S., Kohli, P., Lasenby, J., Leskovec, J., Liu, T.-Y., Manrai, A., … Zitnik, M. (2023). Scientific discovery in the age of artificial intelligence. Nature, 620(7972), 47–60. https://doi.org/10.1038/s41586-023-06221-2 Obsah obrázku socha, umění Popis byl vytvořen automaticky Obsah obrázku socha, umění Popis byl vytvořen automaticky Questions? Jan KLEINER jkleiner@mail.muni.cz Thank you for your attention. Obsah obrázku text, Písmo, Grafika, snímek obrazovky Popis byl vytvořen automaticky