Real-time Spoken Language Understanding for Social Care Robots

The PhD project will investigate and build spoken language understanding (SLU) models from data for a social care robot simulation, to react quickly and effectively to instructions in a home environment. The models built will respond to instructions like "please put the remote control on the table" and follow-up repairs like "no, the left-hand table" when the speaker realizes the robot has made a mistake, in real time and without delay.

To build these SLU algorithms, the project will involve collecting data with pairs of human participants in a Virtual Reality environment. The models built will aim to give the most accurate user intention word-by-word, but also a continuous measure of confidence corresponding to human reaction times. The models will be tested both in the simulation environment and real-world equivalent robots.