Automating Representation Choice for AI Tools

AI engines are ubiquitous in our lives: we talk to our mobile phones, ask directions from our sat-navs, and learn new facts and skills with our digital personal assistants. But most of the time, we need to learn how these systems work first: we have to adapt to them as they are not aware of our level of experience, expertise and preferences. AI engines are fast and can deal with a deluge of data much better than us, but they do so in machine-oriented ways, which are often inaccessible and unintelligible to humans. How can we build machines that will adapt to us?

The aim of this project is to identify and study how humans represent information that they want to work with and from which they will obtain new knowledge. Humans have the capability to choose the representation that works for them to enable them to solve a new problem, and moreover, if the representation needs to be changed, they can spot this and change it. Unlike humans, machines in general have fixed representations and do not have the understanding of the user. For example, sat-nav systems will only give directions with elementary spatial commands or route planning functions, whereas humans give directions in many forms, for instance in terms of landmarks or other geographic features that are based on shared knowledge.

We want to model in computational systems this inherently human ability to choose or change appropriate representations, and make machines do the same. We want to find out what are the cognitive processes that humans use to select representations, what criteria they use to choose them, how we can model this ability on machines, and thus how to engineer systems that will select effective representation to enhance people's problem solving and learning.

This is a grand challenge, because it must marry human with machine capabilities. Our previous EPSRC funded feasibility project brought together an interdisciplinary team to combine expertise in computer science on automated reasoning with diagrammatic representations (Jamnik, Cambridge) with expertise in cognitive science on human problem solving and learning with representations (Cheng, Sussex). This interdisciplinary approach has been critical to the success of the feasibility project.

We previously showed that when humans choose a representation of a problem, they use cognitive and formal properties of the problem and its representation to make their choice. In this project, we build on and generalise this hypothesis and demonstrate its utility by building a mathematics tutor that intelligently picks good representations according to the skill level of different learners. We have the following goals:

1. Develop representation selection theory based on the formalisation of formal and cognitive properties.

2. Develop a cognitive theory to assess the efficacy of alternative representations and methods for selecting representations suited to the competencies of individual users.

3. Devise computational algorithms (software) that mechanise the right choice of representation based on the theoretical foundations.

4. Develop and test the algorithms on a range of domains to demonstrate the scalability and generality of the approach.

5. Build an AI tutoring system that implements automated and personalised representation choice based on the user's level of expertise and experience.

6. Empirically evaluate the capability of the tutoring system to select beneficial representations for supporting problem solving.

Our work is novel in that it will address the problem of appropriate representation choice. Moreover, we will build novel cognitive theories and computational models that will allow AI systems to operate in more human-like ways and adapt to the requirements of the problem and the needs of the user. Thus, the potential impact will span numerous domains where systems interact with humans to represent information and use it for extracting new knowledge.

"Automating Representation Choice for AI Tools" is an ambitious project where we want to find out how humans choose and also change representation during problem solving. We want to develop engines that will give machines the same capability. Such machines will accrue many of the benefits that humans obtain from changing representation, including: more effective communication with a human by selecting a representation that is well-suited to their level of familiarity with the topic; better understanding of a human by identifying and adopting the representation that the human favours; and greater flexibility in adapting to the needs of the user. As such, the project will have economic, societal and knowledge impact.

The impact of this work will be wide as AI systems are becoming ubiquitous. When interacting and exchanging information with humans, developers of such products need to represent information in human understandable ways. We will devise techniques that will help developers of AI systems to build products that choose representations appropriate for their users -- and thus have potential economic benefit to them.

Making machines more human accessible will also benefit the society at large as it will potentially bring technical tools to those that are less technically versed.

We have already outlined how our work will contribute to the expansion of knowledge in Academic Beneficiaries. The techniques we will develop are novel and will provide a scientific advance in all areas (in academia and in industry) that must represent information for solving problems, reasoning, etc.

Whilst we focus on education as our initial concrete application target, our contributions will be general and could apply across different domains. They will lead to better understanding of representations and their relation to human expertise and preferences. As intelligent tools make part of our everyday life, we will all benefit from having more human-like systems that adapt to us, rather than us adapting to them.

Artificial Intelligence is a research area for growth by the EPSRC. The capability to select and adapt representations to suit human needs will provide a novel foundational capability for AI. The EPSRC funded Human-Like Computing (HLC) network recognises the importance of representation selection; the first of the cognitive science challenges in the network's strategy roadmap (https://epsrc.ukri.org/newsevents/pubs/human-like-computing-strategy-roadmap/) asks: "How do we create systems that create and revise their mental representations to fit the problem being addressed?" As a recognition of the importance and the potential impact of our work, the HLC network of interdisciplinary experts fully endorse this project (see Other Attachments).

To achieve maximum impact, we will follow a comprehensive dissemination strategy. We will publish our results in all relevant communities (e.g., artificial intelligence, cognitive science, automated reasoning, diagrams, knowledge representation, human-computer interaction, information visualisation, education) to achieve wide dissemination. We will demonstrate our work by organising a workshop and preparing tutoring material aimed at academic as well as more specific industrial communities. This will also provide a community building opportunity for people interested in representations and more generally, human-like computing. We will continue to participate at outreach activities to raise the awareness of the importance of human-like computing. We will create a web repository to enable free and public access to our papers, corpus of problems and their solutions, software and tutorials. Finally, we have enlisted an Advisory Board of experts spanning all areas relevant to this project, and coming from academia, education and industry - their advice will help us stay focused on relevant problems, influence diverse communities that they lead, and transfer our technology widely.