This paper presents objective metrics for how explainable artificial intelligence (XAI) can be quantified. Through an overview of current trends, we show that many explanations are generated post-hoc and independent of the agent's logical process, which in turn creates explanations with limited meaning as they lack transparency and fidelity. While user studies are a known basis for evaluating XAI, studies that do not consider objective metrics for evaluating XAI may have limited meaning and may suffer from confirmation bias, particularly if they use low fidelity explanations unnecessarily. To avoid this issue, this paper suggests a paradigm shift in evaluating XAI that focuses on metrics that quantify the explanation itself and its appropriateness given the XAI goal. We suggest four such metrics based on performance differences, D, between the explanation's logic and the agent's actual performance, the number of rules, R, outputted by the explanation, the number of features, F, used to generate that explanation, and the stability, S, of the explanation. We believe that user studies that focus on these metrics in their evaluations are inherently more valid and should be integrated in future XAI research.
|Title of host publication||20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021|
|Publisher||International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)|
|Number of pages||6|
|State||Published - 2021|
|Event||20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021 - Virtual, Online|
Duration: 3 May 2021 → 7 May 2021
|Name||Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS|
|Conference||20th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2021|
|Period||3/05/21 → 7/05/21|
Bibliographical notePublisher Copyright:
© 2021 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
- Explainable artificial intelligence
- Human-agent systems
- Interpretable machine learning
- System evaluation