Predictability and Comprehensibility in Post-Hoc XAI Methods: A
User-Centered Analysis

D 2023

Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis

JALALI, Anahid; Bernhard HASLHOFER; Simone KRIGLSTEIN a Andreas RAUBER

Základní údaje

Originální název

Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis

Autoři

JALALI, Anahid; Bernhard HASLHOFER; Simone KRIGLSTEIN a Andreas RAUBER

Vydání

Intelligent Computing, 2023

Nakladatel

Springer

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Utajení

není předmětem státního či obchodního tajemství

Odkazy

URL

Označené pro přenos do RIV

Organizační jednotka

Fakulta informatiky

DOI

https://doi.org/10.1007/978-3-031-37717-4_46

Klíčová slova česky

eXplainable Artificial Intelligence, Machine Learning Interpretability,Human Computer Interaction

Klíčová slova anglicky

eXplainable Artificial Intelligence, Machine Learning Interpretability,Human Computer Interaction

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 5. 9. 2023 22:05, Priv.-Doz. Dipl.-Ing. Dr. Simone Kriglstein

Anotace

V originále

Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users’ ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users’ ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model’s decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users’ understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.

Citovat

JALALI, Anahid; Bernhard HASLHOFER; Simone KRIGLSTEIN a Andreas RAUBER. Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis. In Arai, K. Intelligent Computing. Springer, 2023. Dostupné z: https://doi.org/10.1007/978-3-031-37717-4_46.

@inproceedings{2307097,
   author = {Jalali, Anahid and Haslhofer, Bernhard and Kriglstein, Simone and Rauber, Andreas},
   booktitle = {Intelligent Computing},
   doi = {https://doi.org/10.1007/978-3-031-37717-4_46},
   editor = {Arai, K.},
   keywords = {eXplainable Artificial Intelligence, Machine Learning Interpretability,Human Computer Interaction},
   language = {eng},
   publisher = {Springer},
   title = {Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis},
   url = {https://link.springer.com/chapter/10.1007/978-3-031-37717-4_46},
   year = {2023}
}

TY  - CONF
ID  - 2307097
AU  - Jalali, Anahid - Haslhofer, Bernhard - Kriglstein, Simone - Rauber, Andreas
PY  - 2023
TI  - Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis
PB  - Springer
KW  - eXplainable Artificial Intelligence, Machine Learning Interpretability,Human Computer Interaction
UR  - https://link.springer.com/chapter/10.1007/978-3-031-37717-4_46
N2  - Post-hoc explainability methods aim to clarify predictions of black-box machine learning models. However, it is still largely unclear how well users comprehend the provided explanations and whether these increase the users’ ability to predict the model behavior. We approach this question by conducting a user study to evaluate comprehensibility and predictability in two widely used tools: LIME and SHAP. Moreover, we investigate the effect of counterfactual explanations and misclassifications on users’ ability to understand and predict the model behavior. We find that the comprehensibility of SHAP is significantly reduced when explanations are provided for samples near a model’s decision boundary. Furthermore, we find that counterfactual explanations and misclassifications can significantly increase the users’ understanding of how a machine learning model is making decisions. Based on our findings, we also derive design recommendations for future post-hoc explainability methods with increased comprehensibility and predictability.
ER  -

JALALI, Anahid; Bernhard HASLHOFER; Simone KRIGLSTEIN a Andreas RAUBER. Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis. In Arai, K. \textit{Intelligent Computing}. Springer, 2023. Dostupné z: https://doi.org/10.1007/978-3-031-37717-4\_{}46.

Přehled o publikaci