Start Making Sense: Cognitive and affective confidence measures for explanation generation using epistemic planning

Consider a tourist on a walking tour of a city. After reaching a place where they can see they are almost back to the starting point, their tour guide says "Let's go up that hill", pointing to a large hill. "We can get a good view of the city from there." However, on seeing the tired expression on the tourist's face, the guide adds "Or we can stop in that pub over there and take a break." This scenario has two important features: (1) people like to know what is going on. This is especially true in situations where a decision may not have been anticipated or expected. Here, an explanation may be needed not just to justify the decision but also to establish confidence in that choice-in other words, to trust it; (2) being able to read the situation and adapt to the needs of the moment is important when ranking the possible actions that could be taken. Here, a decision may need to be made dynamically. These two features add up to 'dynamic trust maintenance', and are needed for a broad range of the AI systems that are expected to be deployed in the near future, e.g., automated vehicles, service robots, or interactive voice-based assistants.

This project addresses the need for dynamic trust maintenance (which is not generally available in interactive and autonomous systems) by bringing together experimental research in cognitive science involving cooperative joint action with the practical construction of AI planning tools to apply to the task of explanation generation. This challenge will be addressed through these concrete objectives: (1) to study cooperative joint action in humans to identify the emotional, affective, or cognitive factors that are essential for successful human communicative goals, (2) to enhance epistemic planning techniques with heuristics derived from the cognitive science studies, and (3) to deploy the resulting system to generate human-like explanations and evaluate the effectiveness of the resulting approach with human participants.

Knowledge. The main scientific advances will be new ways to help people interact with planner-based artificial intelligences, and new ways to measure people's trust in such systems. Contributions will include: identifying emotional, affective, and cognitive factors relating to levels of trust in explanations; epistemic planning techniques enriched with heuristics derived from the cognitive science studies; and an empirical evaluation of an implemented system which generates human-like explanations. These have benefits for fields of enquiry including: AI, natural language processing, cognitive science, affective computing, and human computer interaction. The plan is to publish 9 papers in journals and conferences, and to run an expert workshop to help bridge communities.

Society. Social beneficiaries of the project include policy makers who act as intermediaries between technical experts and the general publics, on topics such as the "Right to Explanation". However, the most important "society" impact will ultimately be on public end users empowered by the development of explainable AI. Through our tour guide domain of application, we aim to help people to understand better the strengths and weaknesses of current and future AI, and to benefit from the next generation of intelligent advisors. The plan is to run a co-design workshop, and a public engagement event.

People & Capabilities. This project will contribute to EPSRC's planned network around explainable AI and human-like computing in the UK. We will seek to share our work with researchers at the interface between AI and cognitive science, wherever they are located, helping build a critical mass of researchers engaged with R&D in this area. The project team will be mentored in sharpening their academic profiles, supported in commercialisation, and receive transferrable skills training. To deliver education and inspire the next generation, the team will work with excellent students, especially from the Masters in Design Informatics. The plan is to supervise at least 6 student projects.

Economy We will gain impact via the release of data, software, APIs, and documentation in the public domain under an appropriate license. We will seek to initiate work with an SME project partner with a novel biometric sensor system. Via Informatics Ventures (IV), there is effective entrepreneurship training, coaching for spinoffs, and business development support. The plan is to expose a Design Informatics student cohort (~30 students) to the project ideas, and use IV to support at least 1 entrepreneurial student.