Debating driverless futures: Anticipatory governance and responsible innovation for self-driving cars

Lead Research Organisation: University College London
Department Name: Science and Technology Studies

Abstract

In the middle of the afternoon on May 7th, 2016, near Williston, Florida, Joshua Brown joined the long list of fatalities on the world's roads. However, his death was different. He was his car's only occupant but, as far as we know, he was not driving. His car was in 'Autopilot' mode. The technology in his Tesla Model S that was designed to keep him safe failed to see a white truck that was crossing his carriageway against the bright white sky behind it. Brown's Tesla hit the trailer at 74mph, after which it left the road and hit a post. Had the car veered left instead of right, crossing onto the opposite carriageway, the world's first fatal self-driving car crash could have caused a higher death toll and even greater controversy.

Self-driving cars promise to be one of the most disruptive technologies of the early 21st Century. Enthusiasts for the technology think that it could solve problems such as access to transport for disabled people, traffic jams and hundreds of thousands of deaths on the road each year, most of which are cause by human error. Some companies say they will sell self-driving cars as early as 2018. Governments in the UK and elsewhere see huge potential in securing economic growth and new high-tech jobs for their populations. The UK's Industrial Strategy has prioritised self-driving cars and increased investment in the machine learning technologies that will allow computers to replace humans behind the wheel. Morgan Stanley, an investment bank, forecasts a multi-trillion dollar global market with billions of extra dollars in productivity gains in a 'New Auto Industry Paradigm'. The consultancy firm KPMG calls self-driving cars 'The Next Revolution'.

The typical approach to a new technology is for society to understand its effects only in hindsight. For self-driving cars, this would be a bad idea. Policymakers, innovators and the public risk sleepwalking into a future in which technology worsens inequality and loses public trust. The history of the car in the 20th Century shows us that, while technologies can have enormous benefits, they can also cause harm and lock society into new ways of living that then prove hard to change. For self-driving cars, the question is whether we can develop a more alert approach to the technology as it is emerging, before it becomes part of our everyday lives. Rather than innovation being 'driverless', we should look for ways in which innovators and policymakers can take responsibility for the futures they help create.

To maximise the public benefits of self-driving cars, we should scrutinise innovations and policies that are currently underway. The engineering of our future transport systems is too important to be left to engineers alone. There is a need for democratic discussion of the opportunities and uncertainties of self-driving cars. Rather than guessing at the hopes and fears of consumers and citizens, we should instead ask people what they really think.

In 2017, the House of Lords science and technology committee concluded, "There is a clear need for further Government-commissioned social and economic research to weigh the potential human and financial implications of CAV (Connected and Autonomous Vehicles)." But, while investment in self-driving cars currently totals around $80 billion, there is almost no social science exploring public views about what self-driving cars could mean for the future of transport. This proposal is for the world's first major social science project to bring the public voice into the debate on the future of self-driving cars.

Planned Impact

This project would be the first major UK social science research effort into the uncertainties and possibilities of self-driving car technology. It is designed to be timely and relevant to ongoing policy and technology debates. Self-driving debates have so far been heavily US focussed. However, approaches across different jurisdictions will depend on legacies and cultures of transport. With its strong culture of public transport, the UK has an opportunity to offer an alternative, socially-robust mode of governance. Our project would contribute towards shaping this.

The team's track record in high-impact research is outstanding. Stilgoe has worked for Demos and the Royal Society and advised the UK Research Councils, the European Commission, Nesta and others on emerging technology governance. His framework for Responsible Innovation (with Macnaghten and Owen) has contributed to the shaping of Research Council policy. He has been an expert witness for the Select Committees of both Houses of Parliament. Jones is a leading transport adviser at urban, national and European levels. He is a member of the Independent Transport Commission and the London Roads Task Force. Cohen is an experienced transport consultant and adviser, having worked in the private, public and academic sectors. Parkhurst has been one of the UK's first social researchers to develop work into AVs. Winfield has been a leading voice in early policy debates about the governance of robotics and AI and a pioneer of the emerging field of robot ethics. The team's continued engagement in this debate will be supported by a research and impact assistant, who will be responsible for managing the project's online presence, supporting the project advisory board and engaging with the project's wider network of stakeholders.

The project will make substantial contributions to two areas of policy: first, transport and, secondly, data and machine learning. The project could not be more timely. Both policy areas are changing rapidly and in need of social scientific insight. The 2017 Budget announced plans for cars without drivers on UK roads by 2021. An AVs bill, proposing substantial changes to insurance is currently making its way through parliament. Trials on public London roads are due to begin in early 2019. The National Infrastructure Commission will explore preparing UK roads for driverless cars. The 2017 Budget also announced a Centre for Data Ethics and Innovation. As the General Data Protection Regulation enters law and the UK leaves the European Union, the regulatory landscape will change, creating new questions and new opportunities for social science.

Early policy reports have outlined what is at stake with AVs. The Council for Science and Technology (2015) recommended to the Prime Minister that, "The UK should not just be a playground to test autonomous and connected vehicles without retaining any of the value." David Begg's (2014) 2050 Vision for London concludes that, "automated vehicles have great potential. But we must not allow them to shape our cities in the way the internal combustion engine was allowed to in the last century". A major finding of the House of Lords Science and Technology Committee (2017) was that "There is a clear need for further Government-commissioned social and economic research to weigh the potential human and financial implications of [AVs]".

The approach to maximising impact will be to include stakeholders upstream and throughout the life of the project, in particular through the project advisory group. Representatives from Bosch, CCAV, the Transport Systems Catapult and the StreetWise and Capri trials have agreed to join the group and contribute their expertise to the project (see attached letters of support). The advisory group will help steer the project towards greater relevance for corporate R&D and policy and play a valuable role in promoting project findings.

Publications

10 25 50
 
Description Our first major papers have either been published or submitted. We have made contributions to the following questions:
1. How will self-driving cars depend on and be conditioned by the world (material, social) around them?
2. How can we know if a self-driving car is safe?
3. How might self-driving cars interact with (change, be changed by) the rules of the road?
4. What does public dialogue on self-driving cars tell us about the democratic governance of new technologies?
5. How can we get past seeing the public and their "acceptance" as a problem for new technologies?
Exploitation Route The findings should inform ongoing government policy and technology development.
Sectors Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice,Transport

 
Description Response to Law Commission consultation on AVs
Geographic Reach National 
Policy Influence Type Participation in a national consultation
URL https://driverless-futures.com/2019/02/28/driverless-futures-submission-to-the-law-commission/
 
Description Network on Transport Security and Governance 
Organisation University of Oxford
Department Transport Studies Unit
Country United Kingdom 
Sector Academic/University 
PI Contribution I presented on the theme "Trust as an Indicator of Perceived Risk in Using an Automated Vehicle" to the scoping workshop initiated by the Transport Studies Unit at the University of Oxford..
Collaborator Contribution A scoping workshop was held on 5 March 2020. Each partner made a contribution to the workshop and engaged in discussions about the future network.
Impact The initial outcome is to develop the network around a publication.
Start Year 2020
 
Description Trustworthy autonomous systems 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Jack Stilgoe was a co-investigator on proposals for the EPSRC TAS hub and nodes. None of these won. Stilgoe was invited to join the advisory board for the TAS hub.
Collaborator Contribution Hub proposal was led by Michael Fisher at Liverpool
Impact None yet
Start Year 2019
 
Description Book review in the Guardian 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact A book review published in the Guardian https://www.theguardian.com/books/2020/jul/16/why-we-drive-by-matthew-crawford-review-a-high-speed-reverse-into-nostalgia?CMP=share_btn_tw
Year(s) Of Engagement Activity 2020
URL https://driverless-futures.com/2020/07/16/review-of-why-we-drive-by-matthew-crawford/
 
Description Debating driverless futures 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact As part of social science week, the project held a day of public dialogue at the science museum's Driverless exhibition. We made a film of the day https://driverless-futures.com/2019/12/11/doing-dialogue-at-the-science-museum/
Year(s) Of Engagement Activity 2019
URL https://driverless-futures.com/2019/12/11/doing-dialogue-at-the-science-museum/
 
Description Driverless Futures blog 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The project blog (https://driverless-futures.com/blog/) has received more than 4,000 unique visitors
Year(s) Of Engagement Activity 2019,2020
URL https://driverless-futures.com/blog/
 
Description Driverless Futures launch with transport minister 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Our project launch event had more than 100 people, at Nesta in London, and was introduced by Jess Norman MP, minister for transport
Year(s) Of Engagement Activity 2019
URL https://driverless-futures.com/2019/04/12/launch-mode/
 
Description FT Comment piece 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact John Thornhill, the FT's tech editor, has a comment piece on trust in AI, discussing the recent NTSB investigation of the latest Tesla Autopilot crash. He starts by quoting NTSB chair Robert Summwalt:

"The lessons from this investigation are as much about people as they are about the limitations of emerging technologies,"

He then mentions my argument that "advances in machine learning must be accompanied by social learning." Even though my new book is pretty short, he makes the argument far more concise than I could manage:

what is essential, he suggests, is to create a collective societal capacity to understand emerging technologies and decide on the appropriate regulatory framework. We cannot leave all this to powerful private corporations.
Year(s) Of Engagement Activity 2020
URL https://www.ft.com/content/0e086832-5c5c-11ea-8033-fa40a0d65a98
 
Description Fully Charged Podcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact In this episode Helen Czerski talks to Jack Stilgoe, Associate professor in Science and Technology Studies at University College London, about automotive automation. Yes, it's an exciting technology that will hopefully benefit everyone, but before we get to that point, it has many, many barriers to overcome. Helen talks to Jack about the politics behind it, the ethical nature of if something goes wrong and how this affects us, as a society.
Year(s) Of Engagement Activity 2021
URL https://fullycharged.show/podcasts/podcast-92-helen-czerski-talks-autonomous-cars-with-jack-stilgoe/
 
Description Jack Stilgoe multiple media interviews 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact https://www.nytimes.com/2019/04/26/technology/sri-lanka-social-media.html
https://edition.cnn.com/2020/01/21/middleeast/technology-reduce-traffic-accidents-intl/index.html
https://www.ft.com/content/0e086832-5c5c-11ea-8033-fa40a0d65a98
https://www.bbc.co.uk/programmes/m0009sxr
https://www.bbc.co.uk/sounds/play/m000d84g
Year(s) Of Engagement Activity 2019,2020
 
Description Keynote presentation at AVS 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Jack Stilgoe was invited to give a keynote talk at the Automated Vehicles Symposium, one of the the world's largest AV conferences, with an audience of thousands.
Year(s) Of Engagement Activity 2019
URL https://driverless-futures.com/2019/07/24/doing-dialogue-a-talk-at-avs-2019/
 
Description Policy engagement with DfT 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Tom Cohen and Jack Stilgoe have been invited to take part in a number of policy discussions and workshops with the Department for Transport

20/11/19 DfT Science Advisory committee
9/11/2019 Policy roundtable on public engagement
1/11/2019 CCAV away day
31/7/19 RAC Foundation/DfT policy picnic
24/6/19 CSAP DfT workshop, Cambridge
Year(s) Of Engagement Activity 2019,2020
 
Description Presentation to Bristol Technology Showcase "Will People Share Automated Vehicles?" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact The Bristol Technology Showcase (BTS), held on 8 November 2019, focused on emerging technologies and those themes encompassed in the 4th industrial revolution with a view to examining how they will affect businesses and wider society as a whole. The event was part of an umbrella brand of technology events that were co-located over the same week under the banner of the Bristol Technology Festival. The presentation "Will People Share Automated Vehicles?" contributed to a session titled "The Future of Connected Automated Mobility".
Year(s) Of Engagement Activity 2019
URL https://bristoltechnologyshowcase.com/
 
Description Presentation to ESRC Festival of Social Science "Some Findings from Social Science Investigations into the Implications of Automated Road Transport" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact A presentation to a day-long engagement event organised at the University of Surrey, Guildford within the ESRC Festival of Social Science, 06 November 2019. The focus of the day was on new mobility technologies.
Year(s) Of Engagement Activity 2019
 
Description Public engagement on AVs in Manchester 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact As part of a global citizens' debate on driverless mobility coordinated by Missions Publiques, we worked with Involve, the public participation think tank, and Transport for Greater Manchester to run a day of public dialogue. The report was published by Involve.
Year(s) Of Engagement Activity 2020
URL https://www.involve.org.uk/sites/default/files/uploads/FINAL%20Citizens%27%20Conversation%20on%20Dri...
 
Description Science museum Driverless exhibition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Jack Stilgoe was an advisor to the Science Museum's Driverless exhibition, which began in 2019.
Year(s) Of Engagement Activity 2019
URL https://www.sciencemuseum.org.uk/see-and-do/driverless-who-is-in-control
 
Description Second FT Comment piece 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Tim Harford mentions Who's Driving Innovation in an FT Comment piece
Year(s) Of Engagement Activity 2020
URL https://timharford.com/2021/02/miracle-tech-that-is-anything-but-a-taxonomy-of-bionic-duckweed/
 
Description Self-Driving Cars and the Politics of Innovation - Interview with ICTC 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact On April 8th 2020, ICTC (the Canadian Information and Communications Technology Council) spoke with Dr. Jack Stilgoe, Senior Lecturer in the Department of Science & Technology Studies at University College London, where he researches and teaches the governance of emerging technologies. Dr. Stilgoe is the Principal Investigator of the Driverless Futures? Project, a three-year social science project looking at the governance of self-driving cars.
Year(s) Of Engagement Activity 2020
URL https://driverless-futures.com/2020/05/21/self-driving-cars-and-the-politics-of-innovation-interview...
 
Description Talk at Robosoft: Software Engineering for Robotics, RAEng, London, November 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Alan Winfield gave an invited talk entitled 'Ethical Standards in robotics and AI: Responsible Robotics' at the 2 day meeting RoboSoft: Software Engineering for Robotics, organised by the University of York and hosted at the Royal Academy of Engineering. The talk presented RoboTIPS and the ethical black box; Autonomous Vehicles were also used as an illustrative example.
Year(s) Of Engagement Activity 2019
URL https://www.cs.york.ac.uk/robostar/robosoft/
 
Description The Received Wisdom podcast 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact New podcast on science and technology policy, hosted by Jack Stilgoe and Shobita Parthasarathy
Year(s) Of Engagement Activity 2019,2020
URL https://podcasts.apple.com/us/podcast/the-received-wisdom/id1476334065
 
Description The attachments of autonomous vehicles 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Dec 2020, A conference that was due to be in Los Angeles, but took place on Zoom. This was part of a series of meetings, funded by the US National Science Foundation, on Mathematical Challenges and Opportunities for Autonomous Vehicles. My talk was on the ways in which so-called 'autonomous' vehicles are inextricably attached to various social and technical infrastructures.
Year(s) Of Engagement Activity 2020
URL https://driverless-futures.com/2020/12/15/the-attachments-of-autonomous-vehicles/
 
Description The car. The future. Me. 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact A new exhibition at the British Motor Museum looks at the future of the car. I'm one of the talking heads in Richard Jones's video that accompanies the exhibition.
Year(s) Of Engagement Activity 2020
URL https://driverless-futures.com/2020/07/21/the-car-the-future-me/
 
Description The myth of the autonomous vehicle 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact I was asked to do the keynote talk at the Shift Mobility 2020 conference in Berlin on 3rd September. Via Zoom, natch. I spoke about the seductive myth of autonomous vehicles and how it could lead to bad policy decisions.
Year(s) Of Engagement Activity 2020
URL https://driverless-futures.com/2020/09/04/the-myth-of-the-autonomous-vehicle/