RoboTIPS: Developing Responsible Robots for the Digital Economy

Lead Research Organisation: University of Oxford
Department Name: Computer Science

Abstract

This fellowship will to bring together a variety of people from different walks of life, including academics, industry, civil societies, policy makers and members of the public, in order to create new ways of developing and managing technological innovations. There is often a tension between the economic needs for increasing technological innovation and the ways in which these innovations may be developed responsibly - that is in a manner that is societally acceptable and desirable. We will develop an approach that aims to anticipate not only the positive outcomes but also the potentially negative consequences of technological innovations for society. We will draw on this and an understanding of people's lived rights and obligations to provide creative resources and methods for designers to develop responsible and accountable new technologies. Responsible Innovation lies at the heart of technologies in the Digital economy that aim to promote trust, identity, privacy and security.

Although it has been drawn on in other scientific domains, as yet we have no complete example of how responsible innovation can be successfully applied in the DE sector. The fellowship will consider a motivating example to develop responsible innovation in action. We will look into one particular domain of technology and develop an agile process which will take account of the views of a wide range of people in a fast-changing context, in order to have some influence over the trajectory of an innovation. We will focus on the domain of social robots, those which interact with people and make decisions about what to do on their own accord. Because they make their own decisions in order to perform actions, we need to be able to recover what they did and why they did it, when things seem to go wrong. We will develop an ethical black box (EBB) through which the social robot will be able to explain its behaviour in simple and understandable ways. The development of the EBB will be an example of responsible innovation. We will test this out in particular accident investigations as a social process and we will do this in 3 different study domains. In the final stages of the fellowship, we will show the outcomes of the technological development and the investigations through a variety of means, including through the web and a final public showcase event. This will be to a variety of people including the general public, policy makers, and developers.

Planned Impact

Science:
Key scientific communities will benefit from the knowledge generated by the fellowship and its responsible innovation (RI) approach. This includes fields across robotics, human robot interaction, robot ethics, ethical design, value sensitive design, Human-Computer Interaction, Artificial Intelligence and the social sciences. In particular, the development of the ethical black box (EBB) will form a motivating example of RI that is both human centred and based on a Trust, Rights and Relational approach, and that can generalised to other domains and technologies. The findings of the programme of work and its unique approach will be disseminated through high quality publications and conferences.
Industry:
The fellowship will develop a unique transparency mechanism that that can make robotic decision making open and accountable and thereby foster user trust in social robots across society. This fellowship work will therefore enhance responsibility in industry and advance the acceptance of social robots. The development of the EBB will benefit industry in the specific domains studied in phases 1 and 2 and will also reach a broader range of settings through later activities that will generalise the study findings and RI approach. Impact will be fostered through a series of stakeholder workshops and designer studies and will be supported by the fellowship's industrial partners.

Designers and innovators: The fellowship will benefit designers and innovators by fostering a reflective and inclusive innovation process and developing a Trust, Rights and Relational based approach to design which acknowledges the lived experiences of users and the networks and ecosystems they inhabit. Research activities involving design and innovation stakeholders will deepen understanding of how these professionals currently practice. This understanding will then be drawn on to produce mechanisms through which designers and innovators can embed responsibility within the development of technologies for the digital economy (DE). These mechanisms will include a responsibility toolkit.

Policy: The fellowship will engage critically with the concept of responsibility in the DE and will explore and engage with new forms of anticipatory governance. The perspectives of policy stakeholders will be elicited across the study and the outcomes of this engagement will be combined with other project activities to develop a responsibility toolkit and specify an agile governance approach for social robots. This fellowship will thereby benefit policy makers seeking to identify novel and effective means to regulate the use of social robots and, through the generalisation of the study findings, other innovative technologies.

Society and the general public: The EBB will benefit the wellbeing of citizens in the DE by fostering accountability in the use of social robots and a culture of transparency and responsibility in their use. It will also produce broad societal benefit through the development of a Trust, Rights and Relational approach which will elicit the concerns of citizens and consider how to address them in order to increase trust and forge better relationships to maximise the potential of new digital technologies. The fellowship will include dedicated public engagement and communications activities - to include educational materials, a video animation and participation in public understanding of science events.
 
Description The project findings so far cover a number of key areas:

Requirements for the Ethical Black Box
Through analysis of relevant literature and discussions with industry developers we have begun to identify the requirements for the Ethical Black Box for social robots. These include technical requirements such as compatibility with standard operating systems and formats for data storage, and non-functional requirements regarding the deployment of robots in different social contexts.

Accident/Incident investigation scenarios
We have analysed relevant literature and worked closely with experts in the conduct of accident investigations in aviation and other transport sectors. We have adapted what we have learnt to the domain of social robots and developed an understanding of what is needed in our accident/incident investigation scenarios through which we will test the Ethical Black Box. We have started to work up our first scenario: an accident involving an assistive robot in a social care setting. We have conducted a failure modes analysis to establish a realistic malfunction for the robot and combined this with an analysis of the organisational structure to develop a scenario in which a technical failure combines with social actions to lead to an accident that harms a human user. We have identified a suitable methodology for accident/incident investigation and will shortly be running a mock investigation scenario with research participants.

Practices of developers
We have interviewed companies involved in the design and development of social robots to identify their work practices and their perspectives on safety and responsibility. This has revealed important insights, such as the tendency for products to be developed without a specific use case in mind, meaning that contextual issues of safety are not identified. In addition, the small size of many start up companies mean they do not have staff members dedicated to working on legal compliance and safety issues.

AI regulation
We have taken part in a large number of events involving engagement with stakeholders in social robots and AI more generally. This has helped us to identify the issues of interest and concern to different stakeholder groups. For instance, the designers of automated vehicles and other social robots are aware of ethical concerns regarding automation and are keen for their work to be supported with guidance for good practice. They are particularly interested in guidance on real world dilemmas that emerge when they are developing and testing their technologies - as opposed to high level thought experiments over abstract scenarios.
Exploitation Route One year in, the project is already achieving academic and non academic impact. We expect this to continue and grow significantly as we move into years 2-5.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare,Government, Democracy and Justice,Transport

URL https://www.robotips.co.uk/
 
Description Although the project is at an early stage, it is already serving to produce impact in a number of areas. Research Since the start of the project, we have successfully written papers for: Artificial Intelligence, the conference on Human Robot Interaction, the CHI conference on Human Factors in Computing, and the ACM Halfway to the Future conference. We have also given presentations at a number of research workshops and conferences. Policy In May 2019 the UK All Party Parliamentary Group on Data Analytics produced a report on Trust, Transparency and Technology. The report discussed how to build ethical data policies for the public good and specifically discusses the idea put forward by Alan Winfield and Marina Jirotka of the Ethical Black Box as a mechanism to foster societal trust by investigating incidents and accidents involving social robots. The report recommends the adoption of black boxes in automated vehicles. Through attendance at stakeholder engagement events the project team has also contributed to do the development of policy recommendations. These include the RUSI report 'Data Analytics and Algorithms in Policing in England and Wales: Towards A New Policy Framework' and ongoing work by the Royal Academy of Engineering to discuss the challenges of ethics and bias in AI systems. Industry We are building pathways towards positive impact on industry. We have established partnership in the project with a number of robotics companies - for instance Open Bionics and Consequential Robotics - and discuss matters of good practice in design and development with them. As the project has gone on we have developed further collaborations, for instance with Hewlett Packard and Akara Robotics. We have given presentations and been involved in panel discussions on good practice at a number of events involving industry - for instance the Future of Care workshop organised by the West of England Health Science Network. Education We have one doctoral student working alongside the project and have provided opportunities for two postgraduate students to gain project experience by working directly on tasks in the study. We have also given presentations about the project at outreach events for school children. Arts We have established a collaboration with Neon Dance. Through this we will contribute to the development of a dance piece exploring ideas around loneliness, companionship and belonging incorporating human centered computing and the development of responsible social robots.The resultant work will tour throughout the UK and Japan in 2021. We will also collaborate with Neon Dance on a range of outreach initiatives including working with young people at Swindon Dance, delivering a lunchtime talk at Pervasive Media Studios, Bristol and hosting an event in partnership with Oxford Dance Forum.  Public We have engaged with the public on project issues by participating in a research stand at the 2019 Oxford IF Science and Ideas Festival. We have also written about the project in Inspired Research, the outreach magazine of the Department of Computer Science at Oxford, and set up a project website. We have been invited to give a talk about the project at the 2020 Hay on Wye festival.
First Year Of Impact 2019
Sector Education,Government, Democracy and Justice,Transport,Other
Impact Types Societal,Policy & public services

 
Description EPSRC - UKRI Artificial Intelligence and Public Engagement workshop
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
 
Description Membership of the expert advisory panel of the Topol Review
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact Alan Winfield contributed to the NHS Health Education England review, led by Dr Eric Topol, which led to publication of the final report "Preparing the healthcare workforce to deliver the digital future" - an independent report on behalf of the secretary of state for Health and Social Care, published in February 2019. Alan Winfield was a member of both Robotics & AI and ethics panels.
URL https://www.hee.nhs.uk/our-work/topol-review
 
Description UK All Party Parliamentary Group on Data Analytics report on Trust, Transparency and Technology.
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.policyconnect.org.uk/appgda/news/trust-transparency-tech-report-launch
 
Description Workshop: shaping the state of machine learning algorithms within law enforcement
Geographic Reach National 
Policy Influence Type Participation in a national consultation
Impact Helena Webb attended a workshop run as part of a collaboration between the University of Winchester and the Royal United Services Institute on machine learning algorithmic systems used in the legal system in England and Wales. The workshop brought together academics, regulators, policy makers and criminal justice professionals. it had the particular aim to help establish a framework for the regulation of such algorithmic systems. The outcomes of the workshop formed part of a subsequent report https://rusi.org/publication/occasional-papers/data-analytics-and-algorithms-policing-england-and-wales-towards-new
URL https://rusi.org/publication/occasional-papers/data-analytics-and-algorithms-policing-england-and-wa...
 
Description Written response to Consultation on Automated Vehicle Trialling Code of Practice
Geographic Reach National 
Policy Influence Type Participation in a national consultation
URL https://www.gov.uk/government/consultations/automated-vehicle-trialling-code-of-practice-invitation-...
 
Description Prehension Blooms 
Organisation NEON Foundation
Country Greece 
Sector Charity/Non Profit 
PI Contribution Neon Dance is an internationally renowned company that embraces a diverse and digital population; a place where artist, partners and people can engage in experiences that are experimental and original, yet accessible to all. Adrienne Hart established Neon Dance in 2004 with the aim of connecting audiences with rigorous artists and original thinkers. As Artistic Director, she cultivates an altruistic environment that enables risk taking and collaboration to flourish; developing works that explore technologies and design with the body as the central tenet. Following an initial conversation with Hannah Manktelow, Adrienne Hart was connected with Marina Jirotka, Professor of Human Centred Computing and Helena Webb, Senior Researcher at Oxford University. We discovered many shared interests and believe that the proposed project, exploring ideas around loneliness, companionship and belonging incorporating human centered computing and the development of responsible social robots will be strengthened as a result of our working together. Helena Webb and Marina Jirotka will engage with Neon Dance through meetings and workshops to contribute to the iterative development of the piece.
Collaborator Contribution The resultant work will tour throughout the UK and Japan in 2021. Neon Dance will also lead on a range of outreach initiatives including working with young people at Swindon Dance, delivering a lunchtime talk at Pervasive Media Studios, Bristol, hosting an event in partnership with Oxford Dance Forum and running an event held by the TORCH network at the University of Oxford
Impact Collaboration began in Jan 2020 so no outputs yet
Start Year 2020
 
Description Industry Forum roundtable on 'Restoring trust in digital technology' 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Marina Jirotka was invited to participate in panel discussion to give her views on restoring trust in digital technologies. This was an invitation only event organised by the Industry Forum, a high profile group that seeks to promotes constructive dialogue between public policy makers, industry operating in the UK, and leading commentators. Details of the panel event can be read here http://www.industry-forum.org/event/restoring-trust-in-digital-tech/. Marina introduced ideas about the what it means to trust digital systems and the capacity for responsibility practices that can foster trust. This lead to discussion amongst those present about what kinds of practices can be put in place at the levels of design and policy.
Year(s) Of Engagement Activity 2019
URL http://www.industry-forum.org/event/restoring-trust-in-digital-tech/
 
Description Ethics of AI panel at the Oxford UIDP summit 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Marina Jirotka was a guest speaker on a panel discussing AI ethics. This was a special panel session run at the 2019 Oxford UIDP summit, which brought together high level staff in industry, academia and policy. As part of the panel she spoke about the RoboTIPS project and opportunities for university-industry-policy collaboration in responsible innovation.
Year(s) Of Engagement Activity 2019
URL https://uidp.org/event/oxford_uidp_summit/
 
Description If Oxford Science and Ideas festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Helena Webb participated in the IF Oxford Science and Ideas festival. She joined other members of the Human Centred Computing group to run a stand in a central Oxford shopping centre. She spent the afternoon telling interested members of the public about the UnBias and RoboTIPS projects and listening to their views on the responsible development of technologies.
Year(s) Of Engagement Activity 2019
 
Description Invited talk at AI@Oxford, September 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Alan Winfield gave a short (10 minute) invited talk entitled 'What could possibly go wrong? Why we need Robot Accident Investigations', presenting the RoboTIPS project, on Tuesday 17 September 2019 at AI@Oxford (billed as Oxford's first major international conference on Artificial Intelligence).
Year(s) Of Engagement Activity 2019
URL http://alanwinfield.blogspot.com/2019/09/whats-worst-that-could-happen-why-we.html
 
Description Joint talk at event 'The Future of Care: how can robots and autonomous systems change frontline healthcare', January 2020. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Marina Jirotka and Alan Winfield gave a joint presentation entitled 'Towards Responsible Robotics in Healthcare' at the workshop on The Future of Care, organised by the West of England Academic Health Science Network, held in Bristol on 30 January 2020. The talk defined responsible robotics and outlined the RoboTIPS project, within the context of robotics for assisted living.
Year(s) Of Engagement Activity 2020
URL https://www.weahsn.net/event/robotics/
 
Description Keynote lecture at the Rutherford Space Conference, December 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Alan Winfield gave the invited keynote 'Appleton Lecture' at the 15th Appleton Space Conference on 5 December 2019, at the Rutherford Appleton Labs. The lecture was entitled 'The Ethical Roboticist: a journey from robot ethics to ethical robots', in which RoboTIPS and the ethical black box were featured.
Year(s) Of Engagement Activity 2019
URL https://www.ralspace.stfc.ac.uk/Pages/15th-Appleton-Space-Conference.aspx
 
Description Marina Jirotka: St Cross College talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Marina Jirotka gave a presentation at St Cross College Oxford about the aims and plans for the RoboTIPS study.
Year(s) Of Engagement Activity 2019
 
Description Participation in workshop on The politics of Autonomous Vehicles, December 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Marina Jirotka, Alan Winfield and Helena Webb participated in a workshop organised by ESRC project Driverless Futures? on The politics of Autonomous Vehicles, held at the Royal Academy of Engineering and UCL, 16-17 December 2019. During the workshop Marina Jirotka gave a presentation entitled 'Towards Responsible Innovation in Autonomous Vehicles', in which the aims of RoboTIPS were outlined.
Year(s) Of Engagement Activity 2019
URL https://driverless-futures.com/events/the-politics-of-autonomous-vehicles/
 
Description Responsible Innovation in ICT 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Helena Webb attended a one day workshop on the role of Responsible Innovation in ICT held as part of the CHI 2019 conference. As part of the workshop she described the work of the RoboTIPS project and joined in discussions about embedding responsibility into processes of design and development. The workshop was attended by a number of professional practitioners in design and development as well as researchers and research students. It led to ongoing discussions and ideas for future collaborations.
Year(s) Of Engagement Activity 2019
 
Description Responsible Innovation panel AI@Oxford conference 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Marina Jirotka was invited to talk on an expert panel at the AI@Oxford conference 2019. The panel discussed opportunities to embed responsible innovation into the development of AI supported technologies. Marina Jirotka spoke about the RoboTIPS project as an instance of a project that seeks to foster responsibility in technology across phases of design, development and implementation.
Year(s) Of Engagement Activity 2019
URL https://innovation.ox.ac.uk/innovation-news/events/aioxford-conference/conference-agenda/
 
Description RoboTIPS project website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact We have set up a project website to provide accessible information about the study for all interested audiences. This will be updated regularly.
Year(s) Of Engagement Activity 2019,2020
URL https://www.robotips.co.uk/
 
Description Schools outreach: March2020 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Helena Webb participated in a schools outreach session organised by St Anne's College at the University of Oxford. The session brought over 60 school students (aged 10-11 years old) from a school in South London to spend the day in Oxford. The school was from an economically disadvantaged area and the students were from backgrounds with little familial experience of Oxbridge. Helena gave a taster session to two groups of 15 students to introduce them to her work.
Year(s) Of Engagement Activity 2020
 
Description Talk at Robosoft: Software Engineering for Robotics, RAEng, London, November 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Alan Winfield gave an invited talk entitled 'Ethical Standards in robotics and AI: Responsible Robotics' at the 2 day meeting RoboSoft: Software Engineering for Robotics, organised by the University of York and hosted at the Royal Academy of Engineering. The talk presented RoboTIPS and the ethical black box.
Year(s) Of Engagement Activity 2019
URL https://www.cs.york.ac.uk/robostar/robosoft/
 
Description UK Winter School on Cyber Security 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Marina Jirotka was invited to give a presentation about the RoboTIPS study at this winter school for doctoral researchers.
Year(s) Of Engagement Activity 2020
 
Description University of Oxford student open day 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact Helena Webb attended an open day held by the Department of Computer Science at Oxford. She participated in an open session during which we spoke to prospective students and their parents about working and studying at Oxford, and described the RoboTIPS study as an example of work that is conducted here.
Year(s) Of Engagement Activity 2019