Machine Learning and Adaptation of Domain Models to Support Real-Time Planning in Autonomous Systems

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics


Simulating low-level cognitive behaviour, such as reaction to stimuli, has been a major focus of research and development in the autonomous systems (AS) community for many years. Automated assessment of sensor data, and reactive selection of actions in the form of condition-action pairs, is well developed in robotic and control application areas. In contrast, a characteristic of more intelligent behaviour is the ability to reason with self-knowledge: an AS knows about the actions it can perform, the resources it has, the goals it has to achieve, the current state and environment it finds itself in; and it has the ability to reason with all this knowledge in order to synthesise, and carry out, plans to achieve its desired goals. So for example an unmanned vehicle on the Mars surface might be requested to collect a rock sample at some position, or a spacecraft might be required to take a photograph of some star constellation. These tasks require an AS to generate or be given detailed plans to achieve them. Enabling applications involving AS to have the general ability to generate reliable plans in this manner is a great challenge, because of the difficulty of creating plans fast enough in real-time situations, and the problems in representing and keeping up to date the AS's domain knowledge.

Recently researchers who are working on automatically creating plans (automated planning) have made many breakthroughs, so that now such automated planners are capable of reasoning very efficiently and accurately with detailed representations of knowledge. This has resulted in automated planning software being used within a wide range of applications including fire fighting, elevator control, emergency landing, aircraft repair scheduling, workflow generation, narrative generation, and battery load balancing. In Space applications, scientists at NASA have been developing systems with such technology for the control of autonomous vehicles, and have deployed systems which can plan activities for spacecraft, schedule observation movements for the Hubble Telescope, and control underwater vehicles.

While the development of automated planning has been encouraging, a major problem remains in all these applications, which limits their adaptability, and makes them difficult to maintain and validate: much of the AS's high level self-knowledge, that is knowledge of such things as actions, resources, goals, objects, states and environment, has to be programmed or encoded into the system before its operation (this encoded knowledge is often called a `domain model'). Experience has shown that this encoding involves a great deal of expert time and effort. It also means that if the AS's capabilities change, for example if the preconditions or effects of an action change, then new knowledge describing this must be re-entered into the system by human experts.

This research project seeks to lead the way to overcoming this challenge by enabling the AS to learn and adapt its own domain model. It seeks to discover methods for an AS to acquire knowledge initially, and to maintain and evolve that knowledge via feedback sensing after executing actions. While methods for empowering AS to learn basic reactions, or learn how to classify data, are well established, methods for getting an AS to learn and adapt knowledge of structures such as actions, is not so well developed. The proposers will use their recent research results in this area and research and develop prototype AS which can learn and adapt their domain models. They will demonstrate and evaluate their research using virtual worlds which model real applications.
Title Oil Rig for Simulation and Training in Virtual Reality 
Description A realistic Oil Rig 3D mesh model used by Robert Gordon University (RGU) Oil & Gas Centre in Aberdeen for training purposes and as an educational resource has been provided in the Virtual University of Edinburgh (Vue) hosted OpenSimulator-based OSGrid "Oil Rig" region, as well as on other OpenSimulator-based grids hosted at Edinburgh. The model includes a seabed Blow Out Preventer (BOP) of the type being studied as an application area for emergency standard operating procedure (SOP) automation within the UK EPSRC supported HedLAMP Project (Grant EP/J011800/1) and with the collaboration of Schlumberger Cambridge Ltd. The Oil Rig region was also shown in immersive 3D virtual reality using an Oculus Rift DK2 head mounted display (HMD) to the HEdLAMP project members at their meeting in Edinburgh on 28th November 2014. 
Type Of Art Artefact (including digital) 
Year Produced 2014 
Impact Creating awareness of virtual reality and virtual worlds for realistic simulation and emergency procedure training purposes. 
Description Artificial Intelligence plan representations and planning methods can be used to model complex processes that are highly critical to product and process safety. Examples of this have been demonstrated in oil well blow out preventer emergency procedures. A Massive Open Online Course (MOOC) in Artificial Intelligence Planning was created and presented three times on the Coursera Platform in 2013, 2014 and 2015 reaching over 113,000 registered students. It was used to brief collaborators on the project. The MOOC materials have also been made available under an open licence (CC-BY-NC) for self-guided study at any time via open educational resources at Edinburgh and on YouTube.
Exploitation Route Artificial Intelligence planning techniques are usable in many sectors and will work alongside other emerging AI methods to add capabilities to a wide range of systems. The AI Planning MOOC provides technical material to allow others to understand and make use of the techniques explored on this grant.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Chemicals,Communities and Social Services/Policy,Construction,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Energy,Environment,Financial Services, and Management Consultancy,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Government, Democracy and Justice,Manufacturing, including Industrial Biotechology,Culture, Heritage, Museums and Collections,Pharmace

Description To inform oil well and drilling emergency procedure specification and use. To raise awareness and knowledge about Artificial Intelligence Plannign techniques (via MOOC).
First Year Of Impact 2012
Sector Aerospace, Defence and Marine,Agriculture, Food and Drink,Construction,Creative Economy,Education,Energy,Environment,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections,Retail,Transport
Impact Types Cultural,Societal,Economic

Title H3 - Hybrid Hierarchical Heuristic Planner 
Description A novel artificial intelligence planning tool based on a hybrid combination of hierarchical task network (HTN) and heuristic planning. 
Type Of Material Improvements to research infrastructure 
Year Produced 2015 
Provided To Others? Yes  
Impact Initial release of a novel artificial intelligence planning tool based on a hybrid combination of hierarchical task network (HTN) and heuristic planning. Integrated as a module into Drupal. 
Description AI Planning Applications with Schlumberger Cambridge Research 
Organisation Schlumberger Limited
Department Schlumberger Cambridge Research
Country United Kingdom 
Sector Academic/University 
PI Contribution Modelling and use of emergency procedures for blow out preventer (BOP) operations related to oil well operations.
Collaborator Contribution Provision of documents, procedures, diagrams and expert knowledge related to oil well operations and emergency procedures for such operations.
Impact Models of blow out preventer (BOP) emergency procedures, and demonstrations of the potential for AI plan representations and AI planning to be used in such processes.
Start Year 2012
Description Artificial Intelligence Massive Open Online Course (MOOC) 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Massive Open Online Course (MOOC) on topic of Artificial Intelligence Planning (AI Planning) created and run three times in 2013, 2014 and 2016 on the Coursera platform. Reaching 113,565 registered students who had a wide variety of background and came from approximately 120 countries. Instructors were Prof. Austin Tate and Dr. Gerhard Wickler. The MOOC materials were created in cooperation with six international guest lecturers on specialised topics. The MOOC materials have also been provided to be available on demand in future using a CC-BY-NC licence via the University of Edinburgh media servers, in Open Educational Resources facilities at Edinburgh and via YouTube.
Year(s) Of Engagement Activity 2013,2014,2015,2016,2017