Design Strategies for explainable AI: 12 guiding design principles for human-centered XAI

With the rapid progression in artificial intelligence development and application, there is an increasing demand for AI systems that are transparent and capable of providing clear explanations. The upcoming EU’s AI Act calls for transparent and accessible explanations for users of AI systems to enhance user trust in AI decisions, ensuring that these technologies are used responsibly and ethically while protecting users' rights and fostering public confidence. Significant efforts have been made in the field of Explainable AI (XAI) to create sophisticated algorithms that reveal the internal mechanisms of both opaque and transparent AI systems. These advancements include the development of detailed data visualizations and intricate user interfaces, designed to provide deeper insights and enhance understanding of AI decision-making processes. However, many outputs from explainable AI are primarily designed to aid AI practitioners during the pre-deployment phase, rather than to assist non-technical or general users comprehend AI decisions. As such the need for human-centered XAI, specifically considering the requirements of lay or non-technical users, cannot be understated. Our research examines the gulf of explanation: the degree to which an XAI provides an explanation which is aligned with the mental models and understanding of different users. As part of a larger study which examined users’ mental models of AI systems and mapped outcomes to real world XAI, this phase of our research engaged experts in human computer interaction, AI and data science in an expert evaluation of XAI outputs and give feedback and criteria for how new explanations might be designed for end users.