Enhancing Early-Stage XAI Projects Through Designer-Led Visual Ideation of AI Concepts

The pervasive use of artificial intelligence (AI) in processing users’ data is well documented with the use of AI believed to profoundly change users’ way of life in the near future. However, there still exists a sense of mistrust among users who engage with AI systems some of this stemming from lack of transparency, including users failing to understand what AI is, what it can do and its impact on society. From this, the emerging discipline of explainable artificial intelligence (XAI) has emerged, a method of designing and developing AI where a systems decisions, processes and outputs are explained and understood by the end user. It has been argued that designing for AI systems especially for XAI poses a unique set of challenges as AI systems are often considered complex, opaque and difficult to visualise and interpret especially for those unfamiliar with their inner workings. For this reason, visual interpretations which match users’ mental models of their understanding of AI are a necessary step in the development of XAI solutions. Our research examines the inclusion of designers in an early-stage analysis of an AI recruitment system taking a design thinking approach in the form of 3 workshops. We discovered that workshops with designers included yielded more visual interpretations of big ideas related to AI systems, and the inclusion of designers encouraged more visual interpretations from non-designers and those not typically used to employing drawing as a method to express mental models.