Machine Learning and Adaptation of Domain Models to Support Real-Time Planning in Autonomous Systems

Lead Research Organisation: University of Huddersfield
Department Name: Sch of Computing and Engineering


Simulating low-level cognitive behaviour, such as reaction to stimuli, has been a major focus of research and development in the autonomous systems (AS) community for many years. Automated assessment of sensor data, and reactive selection of actions in the form of condition-action pairs, is well developed in robotic and control application areas. In contrast, a characteristic of more intelligent behaviour is the ability to reason with self-knowledge: an AS knows about the actions it can perform, the resources it has, the goals it has to achieve, the current state and environment it finds itself in; and it has the ability to reason with all this knowledge in order to synthesise, and carry out, plans to achieve its desired goals. So for example an unmanned vehicle on the Mars surface might be requested to collect a rock sample at some position, or a spacecraft might be required to take a photograph of some star constellation. These tasks require an AS to generate or be given detailed plans to achieve them. Enabling applications involving AS to have the general ability to generate reliable plans in this manner is a great challenge, because of the difficulty of creating plans fast enough in real-time situations, and the problems in representing and keeping up to date the AS's domain knowledge.

Recently researchers who are working on automatically creating plans (automated planning) have made many breakthroughs, so that now such automated planners are capable of reasoning very efficiently and accurately with detailed representations of knowledge. This has resulted in automated planning software being used within a wide range of applications including fire fighting, elevator control, emergency landing, aircraft repair scheduling, workflow generation, narrative generation, and battery load balancing. In Space applications, scientists at NASA have been developing systems with such technology for the control of autonomous vehicles, and have deployed systems which can plan activities for spacecraft, schedule observation movements for the Hubble Telescope, and control underwater vehicles.

While the development of automated planning has been encouraging, a major problem remains in all these applications, which limits their adaptability, and makes them difficult to maintain and validate: much of the AS's high level self-knowledge, that is knowledge of such things as actions, resources, goals, objects, states and environment, has to be programmed or encoded into the system before its operation (this encoded knowledge is often called a `domain model'). Experience has shown that this encoding involves a great deal of expert time and effort. It also means that if the AS's capabilities change, for example if the preconditions or effects of an action change, then new knowledge describing this must be re-entered into the system by human experts.

This research project seeks to lead the way to overcoming this challenge by enabling the AS to learn and adapt its own domain model. It seeks to discover methods for an AS to acquire knowledge initially, and to maintain and evolve that knowledge via feedback sensing after executing actions. While methods for empowering AS to learn basic reactions, or learn how to classify data, are well established, methods for getting an AS to learn and adapt knowledge of structures such as actions, is not so well developed. The proposers will use their recent research results in this area and research and develop prototype AS which can learn and adapt their domain models. They will demonstrate and evaluate their research using virtual worlds which model real applications.

Planned Impact

Automated planning and scheduling (APS) is considered a necessary component of systems that are to exhibit intelligent behaviour.

Impact to Knowledge

A domain model (DM) represents reusable knowledge that is required as input by APS systems. DMs are difficult to create, requiring a domain expert and a planning expert. The tools and methods developed in this project will enable the development of formal DMs more rapidly, on a larger scale, and with fewer conceptual flaws. We will disseminate research results at relevant academic conferences, building on Prof McCluskey's excellent links with the knowledge engineering community. The AI planning community will benefit from this research as the increased availability of more diverse DMs will drive new planning research.

Impact to the Economy

The tools and methods developed in this project will make AI planning technology more accessible to other disciplines, enabling software engineers to include sophisticated planners into their systems, such as autonomous vehicles. For example, in multi-vehicle cooperative autonomy the learned DMs provide capability descriptions that can be communicated between the agents; a long range Mars rover will be able to learn new domain knowledge in an unknown environment; to investigate and repair defective infrastructure the DMs will enhance flexibility; and for teleoperation and teleautonomy the results of this project will facilitate autonomous, goal-directed behaviour. Another target area will be autonomic systems for road network support where Prof McCluskey is leading an EU COST action. More generally, we will create a community of DM developers that will enable the UK to gain a competitive advantage in developing autonomous intelligent systems using APS technology.

Impact to Society

The most direct impact to the people living in the UK and in other countries will be achieved through the engagement with and promotion of results to the ISCRAM (Information Systems for Crisis Response and Management), where we will build on Dr Wickler's excellent links to this community. The aim here is to use autonomous intelligent systems to create a safer, more sustainable environment and a more effective disaster response when necessary. More specifically, the results of this project will be in several of the research areas listed in the call, which we will relevant to the ISCRAM community: model-building and learning will be directly addressed as we will learn DMs; planning will be addressed as DMs are a required input for a planning system; situational awareness and information abstraction will be addressed as part of the learning, where sensor data is abstracted to knowledge used by a planner; and verification and validation will be directly addressed as part of the domain analysis.

Impact to People

The impact of the project on people will be twofold. On the one hand, people working as developers of autonomous systems will be able to use automated planning technology as part of their tool suite. On the other hand, people that currently have to operate in dangerous environments, e.g. in a post-disaster scenario, will benefit from more flexible autonomous systems, significantly decreasing the risk for emergency responders.


The results of this project will contribute to knowledge in several academic disciplines, including knowledge engineering, AI planning, and autonomous systems. Several applications of the results of this project, including all the scenarios described in the call, will lead to an economic benefit to the partners. The links with the ISCRAM community should ultimately lead to a safer environment, specifically, more safety for people working in emergency response.
Description Our objectives were to develop a practical domain model language (a formal language embodying the application knowledge model in some industrial area), and techniques for creation/adaptation/translation/validation/learning of models in this language for the purposes of automated planning. We were to prototype these new methods in conjunction with the Industrial partners in the EPSRC AIS programme.

As a result of the funded research, we developed new ways to induce, reformulate and adapt application knowledge models, and to improve the quality of domain independent automated planning engines when applied to them. All the significant developments were published. Some of these developments were embodied in the "KEWI" platform which we created and is available online for community download:

We prototyped these new methods in conjunction with our Industrial partner Schlumberger by encoding knowledge in the domain of drilling, and showing how planning can be performed on this.
Exploitation Route Applications of knowledge-based AI Automated Planning to industry.
Sectors Aerospace, Defence and Marine,Chemicals,Construction,Digital/Communication/Information Technologies (including Software),Energy,Manufacturing, including Industrial Biotechology,Transport

Description The findings of this award are impacted on one of the co-sponsors of the Award - Schlumberger. We worked with them in a confidential application of the research technology to their drilling processes. They have since adopted the general area of the grant (Domain Independent Planning) in their business and recruited academics in the area to work with them on this development. The findings have led to new research and development applying these methods to Automated Road Management, Automated Maintennace for Railway assets, and Automated Manufacture.
First Year Of Impact 2015
Sector Transport
Impact Types Economic

Title KEWI - Knowledge Engineering Web Interface 
Description KEWI is a web interface for collaborative work on the production of knowledge bases for task - based knowledge that can be used by automated planning engines. It is available to DOWNLOAD from the URL below. 
Type Of Technology Webtool/Application 
Year Produced 2014 
Impact Application knowledge regarding one of our industrial partner's work processes has been input and created via KEWI. 
Title PDDL description of Drilling Procedures 
Description Several formal specifications of Drilling Procedures have been encoded for our partner in research Schlumberger. These have been animated in order to derive concrete procedures dynamically. 
Type Of Technology Software 
Year Produced 2013 
Impact The software describes and allows one to analyze some of the processes of the supporting organisation. 
Description Artificial Intelligence Massive Open Online Course (MOOC) 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Massive Open Online Course (MOOC) on topic of Artificial Intelligence Planning (AI Planning) created and run three times in 2013, 2014 and 2016 on the Coursera platform. Reaching 113,565 registered students who had a wide variety of background and came from approximately 120 countries. Instructors were Prof. Austin Tate and Dr. Gerhard Wickler. The MOOC materials were created in cooperation with six international guest lecturers on specialised topics. The MOOC materials have also been provided to be available on demand in future using a CC-BY-NC licence via the University of Edinburgh media servers, in Open Educational Resources facilities at Edinburgh and via YouTube.
Year(s) Of Engagement Activity 2013,2014,2015,2016,2017
Description Regular Schools Visits 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Several times a year I visit Schools and give a talk on Autonomous Intelligent Systems, using general knowledge arising from this EPSRC project.
Year(s) Of Engagement Activity 2016,2017,2018,2019,2020