Model Learning Guided by Natural Language Instructions

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics


The focus of my research is to develop methods to enable improved human-robot interaction through the use of a natural language interface for programming robotic tasks. We will focus on the problem of enabling human instruction giving in settings where the robot and human user do not already have a clear understanding of each other's vocabularies and skills. This calls for innovations on decision theoretic models of interaction and their coupling with language learning.
Building on recent work on interactive and adaptive language learning, and combining these works with other innovations on learning in ad hoc coordination settings, we will devise new models of adaptive language learning that will be specifically focussed on the needs of human-robot interaction. So, we will first begin by implementing an interactive system involving human users working together with a physical robot (the PR2, available in the EPSRC Robotarium Research facility) and use these experimental paradigms to collect and curate a corpus of data that includes information about the task, dialogue and environmental variables. We will then use this to learn new models and through the use of these models, explore new ways in which we can address problems such as learning a language in conjunction with skill adaptation.


10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/N509644/1 30/09/2016 29/09/2021
1788970 Studentship EP/N509644/1 31/08/2016 14/09/2020 Yordan Stefanov Hristov
Description The aim of the project is to look into to the field of Human-Robot Interaction, focusing on methods that allow human experts to speed up learning for robotic agents. The developed framework allows for human teachers to establish a common ground/language with a robot agent which can then be used to teach the agent certain tasks through natural language guidance.

Naturally humans can use multiple modalities of information to communicate an agent certain task-specific knowledge that is crucial for the low sample complexity of the learning process. Thus, the challenge is to bridge the high-level semantic space in which the experts reason with the low-level state/action space in which the agents operate.
Exploitation Route The outcomes of the project can be used as a foundation for future research projects in the context of interactive task learning with a robot learner.
Sectors Digital/Communication/Information Technologies (including Software)