A Robot Training Buddy for adults with ASD

Lead Research Organisation: Heriot-Watt University
Department Name: S of Mathematical and Computer Sciences

Abstract

Autism Spectrum Disorder (ASD) affects 695,000 people in the UK, and about 547,000 of these are 18 or over (1.3% of the adults in working age). The unemployment rate among adults with an ASD is higher than 85%, nearly double the unemployment rate of 48% for the wider disabled population. One reason for this is that people with an ASD struggle to interpret social signals, those expressive behavioural cues through which people manifest what they feel or think (facial expressions, vocalisations, gestures, etc.). This project will develop a Socially-Competent Robot Training Buddy that will help adults with ASD to better deal with social signals in work-related scenarios.

The project is inherently interdisciplinary and falls in the new research area of Socially Assistive Robotics, at the crossroads between robotics, psychology, and social signal processing. So far, autonomous robots have largely been seen as functionally engineered to carrying out well-defined tasks in an efficient manner. However Socially Assistive Robots (SARs) must fit into normal human social environments and follow interaction rules that do not disrupt an office or a home or upset their human interaction partners. This project will focus on high-functioning adults with an ASD, and just as physically assistive robots enable people to make movements that are difficult because of physical impairments, the SAR of this project enables people with an ASD to perform social tasks that are difficult - if not impossible - due to social cognition impairments.

The main goal is to reduce the cost of Behavioural Skills Training (BST) through the development of a Robot Training Buddy. BST is recognized as one of the most effective approaches to alleviate ASD effects, but cannot be applied extensively because it is labour intensive. Using an autonomous robot would reduce the human effort and cost of BST and make it more widely available. The main technological challenge is the development of a novel affective architecture that makes a robot suitable for behaviour rehearsal, a critical stage of BST. In behaviour rehearsal, the robot must reinforce the use of appropriate social signals by its human interaction partner while inhibiting the use of inappropriate ones. The team will work with stakeholders involved with training for adults with an ASD to develop workplace-relevant scenarios in which to develop and evaluate the Training Buddy with end-users.

This work will develop the necessary scientific basis for the introduction of socially-competent robots into human social environments, opening the way to a multitude of domestic, educational and assistive applications.

Planned Impact

The following specific impacts are associated with this project:

1.An improved training approach for adults with high-functioning ASD that might eventually lead to an increase in employment opportunities. The use of a robot would enable the more widespread application of behavioural skills training in social signal perception since the availability and cost of trained role-play therapists is a major bottleneck. Partnership in this project by Autism Initiatives provides an important channel for this impact as does involvement of a variety of stakeholders in the project Steering Committee.

2.Development of understanding in public policy makers of the usefulness of social robots for training. The technology and ASD symposium proposed is a specific channel for impacting the policy of the Scottish Government and raising policy-maker understanding of the use of social robots in training applications.

3.Changes in the public perception of robots from seeing them as threatening towards understanding their potential social usefulness. The public engagement activities of the project will help the public to arrive at a more realistic understanding of the use of robots rather than fearing them as sources of job loss or as a generalised threat to humanity.

4.Increased uptake by robot manufacturers of social signal generation and processing in their products
New generations of social robots are just beginning to be built, such as the Aldebaran pepper robot. The project will demonstrate the usefulness of robust social signal processing in social robots and thus promote the incorporation of this technology into these new products. the presence of Aldebaran on the Steering Committee is a channel for this type of technology transfer.

Publications

10 25 50
publication icon
Aylett R (2021) An Architecture for Emotional Facial Expressions as Social Signals in IEEE Transactions on Affective Computing

publication icon
McKenna P (2017) Social Robotics

publication icon
McKenna P (2020) "Sorry to Disturb You"

publication icon
McKenna P.E. (2018) Robot expressive behaviour and autistic traits in Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

 
Title A SoCoRo comic 
Description Along with a SMR producing scientific comics we generated a two-sided comic explaining the topic and showing the robot 
Type Of Art Image 
Year Produced 2018 
Impact Used on open days and University outreach with robots 
 
Description 1. We have designed and implemented humanoid but non-human robot facial expressions and established which of these are reliably recognised by non-diagnosed participants as expressions of approval and disapproval 2. We have developed an approach to testing a system for high-functioning adults with an ASD that uses the Baran-Cohen AQ (Autism Quotient) questionnaire this allowing software to be tested initially on a non-diagnosed set of participants 3. We have designed an experiment to test the impact of interruptions in a workplace related scenario
Exploitation Route Our methodological advances are applicable to other work in social robotics and especially to work with adults with autism
Sectors Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice

 
Description As examples of non-threatening and indeed socially useful robot applications, countering public alarm about the impact of robotics research
First Year Of Impact 2017
Sector Education
Impact Types Cultural

 
Description Expressive behaviour as a social signal for socially-competent human-robot interaction: Expre-ss
Amount £15,869 (GBP)
Funding ID EP/T013737/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2020 
End 02/2021
 
Description Test Play - Developing and testing a new computer role play programme as a research, clinical and educational tool.
Amount £50,000 (GBP)
Organisation Heriot-Watt University 
Sector Academic/University
Country United Kingdom
Start 04/2019 
End 09/2019
 
Description Autism Initiatives 
Organisation Autism Initiatives
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Sharing on-going results of the project
Collaborator Contribution Took part in one-year review of project as part of Steering group and contributed to scenario development for future experiments
Impact Inout to on-going internal experiment design work
Start Year 2016
 
Description One month research visit to University of Geneva 
Organisation University of Geneva
Department Emotion, Elicitation and Expression (E3 Lab)
Country Switzerland 
Sector Academic/University 
PI Contribution Models of relationship between emotions and social signals
Collaborator Contribution Theoretical accounts of emotions and social signals
Impact Clarification of models and theories
Start Year 2020
 
Description Article about the project in The Daily Mail 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact A press release secured an article in the Daily Mail about the project
Year(s) Of Engagement Activity 2017
URL https://www.pressreader.com/uk/scottish-daily-mail/20171017/281788514291214
 
Description Article in BBC Science Focus Magazine 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Invited article in BBC Science Focus Why Robots will not take over any time soon..
Year(s) Of Engagement Activity 2022
URL https://www.sciencefocus.com/future-technology/future-robots-society/
 
Description Article in Heriot-Watt Newsletter 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Media (as a channel to the public)
Results and Impact Article on the project in the University Newsletter which goes to all staff and also to alumni
Year(s) Of Engagement Activity 2017
URL https://www.hw.ac.uk/about/news/cutting-edge-robot-could-reduce-autistic.htm?utm_source=newsletter&u...
 
Description Article in Sunday Herald 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact Article in Scotchs Sunday Herald on the project
Year(s) Of Engagement Activity 2018
URL https://www.heraldscotland.com/news/16222183.how-robot-research-could-help-save-humans-from-themselv...
 
Description BBC Blue Peter appearance 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact SoCoRo CoI Thusha Rajendra appeared on Blue Peter with the project robot, Alyx
Year(s) Of Engagement Activity 2017
URL http://www.bbc.co.uk/programmes/b0911n63/credits
 
Description De of robot at UK Parliament open day 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The Edinburgh Centre for Robotics organised an open day hosted by the UK parliament at which we demoed our robot
Year(s) Of Engagement Activity 2017
 
Description Demo for DHI visit 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact SoCoRo demo for the Digital Health and Care Institute visit
Year(s) Of Engagement Activity 2018
 
Description Demo of system at the Scottish parliament open evening 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The Scottish Parliament hosted an open evening organised by the Edinburgh Centre for Robotics, of which we are members, attended by MSPs on robot technology. We ran a demo and answered questions.
Year(s) Of Engagement Activity 2017
URL https://www.edinburgh-robotics.org/news/201711/year-robotics-parliamentary-reception%C2%A0
 
Description Dynamic Earth demofest 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact We demoed our robot at the annual Demofest organised by the Scottish Informatics and Computer Science Alliance (SICSA) to present ongoing research to Industry and Commerce, held in a large public venue, the Dynamic earth museum, in Edinburgh
Year(s) Of Engagement Activity 2017
URL http://www.sicsa.ac.uk/knowledge-exchange/industry-collaboration/demofest/
 
Description Edinburgh Fringe Festival Show by PI 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact PI Aylett presented The Robots Are Coming - Or Are They? as part of the Cabaret of Dangerous ideas in the Edinburgh Fringe.
Year(s) Of Engagement Activity 2017
URL http://codi.beltanenetwork.org/the-robots-are-coming/
 
Description German newspaper article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article in the Frankfurter Allgemeine about the project with a small video in the online version
Year(s) Of Engagement Activity 2018
URL http://www.faz.net/aktuell/beruf-chance/beruf/autismus-in-der-arbeitswelt-wie-guckt-der-chef-wenn-er...
 
Description German publication of SoCoRo interactive questionnaire 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Nadine Bos from Faz.de published an interactive online version of the SoCoro emotion recognition experiment, in German
Year(s) Of Engagement Activity 2018
URL https://www.faz.net/aktuell/beruf-chance/beruf/autismus-in-der-arbeitswelt-wie-guckt-der-chef-wenn-e...
 
Description HRI Workshop on Explainable Robotic Systems 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk on "Robot-assisted social skills training for adults with ASD"
Year(s) Of Engagement Activity 2018
 
Description Interview with BBC Midlands 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact 10 min interview on the BBC Midlands radio station
Year(s) Of Engagement Activity 2021
 
Description Invited presentation at Shonan symposium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Invited talk on trust and Empathy in robotic systems for the support of the elderly
Year(s) Of Engagement Activity 2018
URL http://shonan.nii.ac.jp/shonan/blog/2017/01/19/multimodal-agents-for-ageing-and-multicultural-societ...
 
Description Invited seminar at University of Lisbon 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited seminar on the SoCoRo project at GAIPS, University of Lisbon
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Christchurch University NZ 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited seminar to the human-robot research group at Christchurch University
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Epi-Rob workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Rajendran, G. (September, 2018). HRI as privileged window into developmental conditions. Invited workshop. ICDL-EpiRob 2018.
Year(s) Of Engagement Activity 2018
 
Description Invited talk, ACM International Conference on Intelligent Virtual Agents 2022 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk to a conference of 100+ people in Portugal
Year(s) Of Engagement Activity 2022
URL https://ivaconference2022.ualg.pt/program/invited-speakers/
 
Description Invited talk, University of Auckland 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited seminar at the robotics group of University of Auckland
Year(s) Of Engagement Activity 2018
 
Description Online interview on Owen Jones show 'Will robots take over the world?' 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact 30 mins interview on robotics on the online Owen Jones show
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=Gs2k4CSbq_g
 
Description Online seminar for University of Geneva 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact 50 researchers attended an online seminar Apr 1st at University of Geneva: title From facial expressions to social signals for useful embodied social agents
Year(s) Of Engagement Activity 2020
URL https://www.campusbiotech.ch/en/node/676
 
Description Panel/talk : Art of Possible - The future of emerging tech in business - opportunities vs threats 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact An online panel and keynotes organised by Technology Scotland to update industry about the impact of AI technologies
Year(s) Of Engagement Activity 2022
URL https://www.youtube.com/watch?v=jqNoRv-Ya9c
 
Description Participation in BEIS video 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Participation in a BEIS video highlighting the research being carried out in Scotland in line with the govt's Industrial Strategy.
Year(s) Of Engagement Activity 2018
 
Description Participation in Demofest 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact DemoFest runs annually at Dynamic Earth ion Edinburgh and acts as a showcase for academic research to regional industry.
Year(s) Of Engagement Activity 2018
URL https://www.sicsa.ac.uk/knowledge-exchange/industry-collaboration/demofest/
 
Description Pint of Science talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact PI Ruth Aylett gave a talk as part of the Pint of Science series run nationally in 2017
Year(s) Of Engagement Activity 2017
URL https://www.hw.ac.uk/research/engage/case-studies/pint-of-science.htm
 
Description Pint of Science talk by team member Frank Broz 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Team member Frank Broz presented in the 2017 Pint of Science series
Year(s) Of Engagement Activity 2017
URL https://www.hw.ac.uk/research/engage/case-studies/pint-of-science.htm
 
Description Podcast fro New Book network 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Podcast with my colleague Patricia Vargas on the New Book Network
Year(s) Of Engagement Activity 2021
URL https://newbooksnetwork.com/living-with-robots
 
Description Poster & talk at EU cog network meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk/poster at the 2018 annual meeting of the EUCog network of excellence
Year(s) Of Engagement Activity 2018
URL https://www.eucognition.org/index.php?page=2018-hamburg-general-info
 
Description Presentation at ECR Gateway event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Rajendran, G. (2018, 4 May). HRI from a psychologist's perspective. Gateway Event. Edinburgh Centre for Robotics. Invited.
Year(s) Of Engagement Activity 2018
 
Description Presentation at Scottish Autism conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact The Socially Competent Robot (SoCoRo) team presented by invitation at the 2018 Scottish Autism Conference. The event, hosted by the Edinburgh International Conference Centre (EICC) on 28th March, gave researchers the opportunity to openly discuss the SoCoRo project aims, and gather feedback from a variety of interested parties: parents of autistic children, academics, teachers, and adults with Autism Spectrum Disorder (ASD).
Year(s) Of Engagement Activity 2018
URL https://www.edinburgh-robotics.org/news/201804/socially-competent-robot-socoro-team-attend-scottish-...
 
Description Press articles in The Scotsman 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact A press release secured an article in The Scotsman and its sister paper The Edinburgh Evening News about the work of the project.
Year(s) Of Engagement Activity 2017
URL http://www.scotsman.com/future-scotland/tech/edinburgh-robot-could-help-autistic-adults-find-work-1-...
 
Description Robot demo 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact HWU Alumni Reunion demo
Year(s) Of Engagement Activity 2018
 
Description Robot demo 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Demo for 10 high-level members of a delegation from China
Year(s) Of Engagement Activity 2018
 
Description Robot demo at ICMI 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The Alyx robot head was demo-ed in the demos session of the International Conference on Multi-modal Interaction (ICMI 2017)
Year(s) Of Engagement Activity 2017
URL https://icmi.acm.org/2017/index.php?id=demos
 
Description Seminar for Edinburgh Napier 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Invited seminar to Edinburgh Napier Robotics research group 19 Jan 2022
Year(s) Of Engagement Activity 2022
 
Description SoCoRo end of year review 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact SoCoRo end of year review for stakeholders, policy makers, and interested researchers
Year(s) Of Engagement Activity 2019
 
Description Team member Franz Broz presented at Art of Possible 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Frank Broz reported the SoCoRo facial expressions experiments as part of a presentation at the Technology Scotland 'Art of Possible' in Glasgow.
Year(s) Of Engagement Activity 2018
URL http://www.enigmapeople.com/blog/art-possible-rise-robots/
 
Description Two team members presented at the EICC Innovation Robotics event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact PI Aylett and CoI Rajendra presented at the Edinburgh International Conference Centre in an innovation Robotics event, covering Robots That Care; and What do robots tell us about ourselves?
Year(s) Of Engagement Activity 2017
URL https://www.youtube.com/watch?v=XIoIOhk2s_s&feature=youtu.be
 
Description Two-day study run with robot at Glasgow Science Centre 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact We ran a two-day live study in the Glasgow Science Centre recruiting 44 participants from visitors during those days. Many bother people observed the experiment as it ran.
Year(s) Of Engagement Activity 2017
URL https://www.hw.ac.uk/about/news/internal/social-robots-creating-a-more-inclusive.htm
 
Description Video interview with Reuters 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Video interview with Reuters about the project
Year(s) Of Engagement Activity 2017
URL https://uk.reuters.com/video/2017/12/27/social-robot-could-help-autistic-adults?videoId=376656477&vi...
 
Description Virtual seminar in Talking Robotics series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Seminar presentation in the Series 'Talking Robotics' Nov 5th 2021
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=zOXKr5-ik4w
 
Description Workshop at IVA2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Workshop on Graphical and Robotically Embodied Agents for Therapeutic Systems - GREATS18 0 held at IVA2018
Year(s) Of Engagement Activity 2018
 
Description Workshop on Socially Assistive Robots and Technology for Healthy Aging 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact "Social and Assistive Robotics"
Presentation to a group of ~50 Scottish researchers and members of govt and healthcare organisations
Year(s) Of Engagement Activity 2018
 
Description Year 1 review meeting with stakeholders 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The year 1 review involved a set of stakeholders including the project steering committee members and an attendee from the Scottish Government
Year(s) Of Engagement Activity 2018
URL https://www.hw.ac.uk/about/news/internal/socially-competent-robot-team-invite-you-to.htm