MaDrIgAL: MultiDimensional Interaction management and Adaptive Learning

Lead Research Organisation: Heriot-Watt University
Department Name: S of Mathematical and Computer Sciences

Abstract

As tech giants like Google, Facebook, Apple and Microsoft continue to invest in speech technology, the global voice recognition market is projected to reach a value of $133 billion by 2017 (companiesandmarkets.com, 2015). Speech-enabled interactive systems in particular, such as Apple's Siri and Microsoft's Cortana, are starting to show significant economic impact, with the virtual personal assistant (VPA) market estimated to grow from $352 million in 2012 to over $3 billion in 2020 (Grand View Research, 2014).

Although such commercial systems allow consumers to use their voice in interacting with their devices and services, the user experience is still limited due to the lack of naturalness of the conversations and limited social intelligence of the VPA. Moreover, the quality of these user interfaces relies on large, carefully crafted rule sets, making development labour-intensive and not scalable to new application domains. With the emergence of the Internet of Things and voice control in the smart home, there is a huge demand for scalable development of natural conversational interfaces across task domains.

MaDrIgAL will develop a radically new approach to building interactive spoken language interfaces by exploiting the multi-dimensional nature of natural language conversation: in addition to carrying out the underlying task or activity, participants in a dialogue simultaneously address several other aspects of communication, such as giving and eliciting feedback and adhering to social conventions. In analogy to the singing voices in a madrigal, simultaneous processes for each dimension operate in harmony to produce multifunctional, natural utterances. Consider the two alternative responses S2a and S2b in the following example:

U1: Hello, I would like to book a flight to London.
S2a: Which date did you have in mind?
S2b: Ok, flying to London on what date?

Whereas S2a only asks for the next piece of information to book the flight (uni-dimensional), S2b also gives feedback about the arrival city, allowing the user to correct any recognition errors (multi-dimensional). We aim to develop a principled multidimensional modelling and learning framework that covers a wide range of different phenomena, including the implicit confirmation in S2b.

This multi-dimensional approach will not only allow us to build systems that support more natural and effective interactions with users, but also enables cost-effective development of such interfaces for a variety of domains by learning transferable conversational skills (e.g., selecting actions in domain independent dimensions). We will therefore demonstrate our approach by building interactive spoken language interfaces for multiple application domains in a home automation scenario, allowing users to interact with for example their Smart TV or heating control system. We will closely collaborate with the industrial partner SemVox to explore this scenario.

The project will bring together expertise in statistical machine learning approaches to state-of-the-art spoken dialogue systems and natural language generation, as well as linguistic theories of multi-dimensional dialogue modelling (collaborating in particular with academic partner Prof. Bunt). MaDrIgAL will develop Next Generation Interaction Technologies relevant to Health Technology and Assisted Living, as well as tackle the question of a common user interface to the Internet of Things and Big Data.

Planned Impact

From an end-user importance point of view, this work will be of interest to a wide range of businesses and companies in the UK, operating in areas such as speech technology, AI, home automation, etcetera. Specifically, we will develop and release a new spoken dialogue system (SDS) architecture compliant with the recently developed ISO standard for dialogue act annotation. This will promote interoperability, enabling the developer community to collaborate and reuse each other's components more easily, and ultimately help industry in developing products with spoken language interfaces more efficiently. The proposed research will also help to strengthen the impact of statistical SDS. Development costs for statistical techniques seem currently too high for manufacturers, due to the requirement of sufficient domain-specific data. The proposed new modular architecture indirectly addresses the economic feasibility of developing products with interactive interfaces for a variety of application domains.

This will ultimately help the UK to push past the US, which is still the main competitor in this area, despite the emergence of new language technology companies such as VocalIQ and Arria. For example, a senior speech scientist at Apple has said publicly that statistical methods will form the algorithmic basis for Siri in the near future. These increased capabilities will also enable us to tackle more complex interaction scenarios, such as social robotics, situated multimodal dialogue with smart devices, and eventually controlling and managing the Internet of Things.

From a societal importance point of view, this project will create a unified user interface which allows every-day users to access and control the Internet of Things in an intuitive way using natural language, and as such lowers the barrier to access and benefit from new technology. This is especially relevant for elderly and/or disabled users in a home automation scenario for example.
 
Description We were selected as the only UK university to participate in the Amazon Alexa Prize Challenge 2017 and again in 2018. This allows us to test current approaches to open-domain dialogue at scale, i.e. with a large number of customers instead of previous lab-based or crowd-sourcing experiments. We found that the real-world use of current neural methods is limited, since they are hard to control. However, we developed a framework combining data-driven as well as rule-based systems, which achieved an impressive 3rd place in this prestigious challenge. We furthermore investigated output generation in task-based dialogues by organising the E2E NLG challenge. We found that current neural approaches are promising since they reduce the need for semantically aligned data. However, they often fail to correctly express a given meaning representation if they lack a strong semantic control mechanism applied during decoding. Moreover, neural models are often outperformed by hand-engineered systems in terms of overall quality, as well as complexity, length and diversity of outputs. In addition, we have build and evaluated a multi-dimensional dialogue manager and submitted the results for review. We find that multi-dimensionality helps to reduce data needs for new applications, since it allows to transfer learnt behaviour from other domains.
Exploitation Route We are currently collaborating with a number of companies, including AdeptMind.ai, EmoTech LTD, Adobe and Amazon.com. I have also developed 2 new MSc programmes in Artificial Intelligence, which offer a specialism in Conversational AI. One of these programmes is aimed at students from other disciplines, such as psychology or engineering, who want to gain additional skills in AI.
We have also set up a spin-out company which benefits from some of the expertise developed in this research.
Sectors Digital/Communication/Information Technologies (including Software),Education,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Retail

URL https://sites.google.com/view/nlplab/
 
Description Heriot-Watt University is currently involved in several knowledge transfer collaborations with industry, including EmoTech LTD, AdeptMind.ai, Adobe and Amazon.com, where we co-create and develop the methods proposed in this grant. For example, we were selected to participate in the prestigious Amazon Alexa challenge, which gives us a great test bed for our methods. Furthermore,Heriot-Watt University has also created a new MSc programme in AI with Speech and Multimodal Interaction, where methods and techniques developed as part of this proposal are taught to students in new courses, such as F20/21CA Conversational Agents. Finally, methods and expertise developed as part of this grant have enabled the creation of our spin-out company Alana AI.
First Year Of Impact 2017
Sector Digital/Communication/Information Technologies (including Software),Education
Impact Types Economic

 
Description DATAIA scientific advisory board
Geographic Reach Europe 
Policy Influence Type Participation in a advisory committee
URL http://dataia.eu/
 
Description GOV.UK Centre for Data Ethics and Innovation - Contributor to Report on Smart Speakers and Voice Assistants
Geographic Reach National 
Policy Influence Type Gave evidence to a government review
Impact GOV.UK Centre for Data Ethics and Innovation - Contributor to Report on Smart Speakers and Voice Assistants
URL https://www.gov.uk/government/publications/cdei-publishes-its-first-series-of-three-snapshot-papers-...
 
Description Member of the RSE Working Group on AI
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
 
Description New MSc Programme in Speech and Multimodal Interaction
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact Verena Rieser created a new postgraduate MSc programme at Heriot-Watt, which aims to educate highly employable experts in creating conversational multimodal interfaces. The programme recently received 6 fully funded studentships by the DataLab/ Scottish funding council.
URL http://www.macs.hw.ac.uk/cs/pgcourses/aiws.htm
 
Description UKRI Closed Roundtable on Next Generation AI
Geographic Reach National 
Policy Influence Type Participation in a national consultation
 
Description AI for Good
Amount £15,000 (GBP)
Organisation Nesta 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2020 
End 09/2020
 
Description Amazon Alexa Challenge 2017
Amount $100,000 (USD)
Organisation Amazon.com 
Sector Private
Country United States
Start 11/2016 
End 11/2017
 
Description Amazon Alexa Challenge 2018
Amount $250,000 (USD)
Organisation Amazon.com 
Sector Private
Country United States
Start 02/2018 
End 12/2018
 
Description DataLab MSc scholarships
Amount £36,000 (GBP)
Organisation Government of Scotland 
Department Scottish Funding Council
Sector Public
Country United Kingdom
Start 09/2017 
End 08/2018
 
Description DataLab knowledge exchange UK Industry
Amount £114,000 (GBP)
Organisation Government of Scotland 
Department Scottish Funding Council
Sector Public
Country United Kingdom
Start 12/2016 
End 12/2017
 
Description EPSRC Impact Acceleration
Amount £45,000 (GBP)
Organisation Heriot-Watt University 
Sector Academic/University
Country United Kingdom
Start 11/2017 
End 10/2018
 
Description direct industry funding
Amount £88,000 (GBP)
Organisation AdeptMind Inc 
Sector Private
Country Canada
Start 09/2017 
End 08/2020
 
Title Alana Chatbot for Amazon Alexa US 
Description We have created an open-domain chatbot called "Alana", which participated as on of the bots in the Amazon Alexa Challenge 2017. 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact As part of this challenge, Alana reached over 360.000 of Amazon Alexa users in the US. Also, Amazon's research tool "CoBot" was largely inspired by our bot's architecture. While the tool is not yet available to the general public, we published a research paper providing details. 
URL https://s3.amazonaws.com/alexaprize/2017/technical-article/alana.pdf
 
Title E2E NLG Challenge: Benchmarking Neural vs. Handcrafted Approaches for Language Generation 
Description Recent end-to-end data-driven Natural Language Generation (NLG) systems are promising to significantly reduce the need for annotated data and ultimately result in reduced development costs for NLG systems. We developed and organised the first shared task on end-to-end (E2E) NLG, which aims to assess whether these novel approaches can generate more complex output which would be suitable for real-world applications. Our shared task received 62 submissions with diverse system architectures by 17 institutions from 11 countries, with about 1/3 of these submissions coming from industry. We consider this level of participation an unexpected success, which underlines the timeliness of this task. 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact We compare 62 submitted systems overing a wide range of approaches, with sequence-to-sequence (seq2seq) models being the most frequently used. We find that seq2seq systems generally score high in terms of word-overlap metrics and human evaluations of naturalness. However, they often fail to correctly express a given meaning representation if they lack a strong semantic control mechanism applied during decoding. Moreover, seq2seq models are often outperformed by hand-engineered systems in terms of overall quality, as well as complexity, length and diversity of outputs. 
URL http://www.macs.hw.ac.uk/InteractionLab/E2E/
 
Title RankMe: Reliable Human Ratings for Natural Language Generation 
Description We have developed and published a new method for evaluating Natural Language Generation. In particular, the RankME method was shown to produce more reliable human ratings for NLG and related tasks. 
Type Of Material Improvements to research infrastructure 
Year Produced 2018 
Provided To Others? Yes  
Impact This method allows to get more reliable results regarding the quality of language generation systems. RankMe was developed for the E2E NLG task, which is a shared international challenge to benchmark end-to-end generation approaches. This methodology is currently also tested for open domain response generation in dialogue systems. A future version will be implemented as part of the Facebook Parl.AI platform. 
URL http://www.macs.hw.ac.uk/InteractionLab/E2E/
 
Title E2E NLG Challenge Evaluation Files 
Description This dataset contains outputs of 56 systems and raw human ratings for 21 primary systems participating in the E2E NLG Challenge. The systems have been trained and tested on the E2E NLG Dataset to generate restaurant recommendation from a flat meaning representation (MR; attribute-value sets). The data consists of system outputs for each MR on the E2E NLG Challenge test set, plus relative human rankings of overall output quality and output naturalness on randomly sampled groups of 5 system outputs for the same MR. The size is 56 * 630 system outputs, with 2,979 5-way rankings of quality and 4,239 5-way rankings of naturalness. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact This dataset was used as a basis to prepare the following papers: - "Findings of the E2E NLG Challenge" (Dušek et al, INLG 2018) - "Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge" (Dušek et al, 2019, submission for the Computer Speech and Language journal). 
URL http://www.macs.hw.ac.uk/InteractionLab/E2E/
 
Title The E2E Challenge Dataset 
Description The E2E dataset is a new dataset for training end-to-end, data-driven natural language generation systems in the restaurant domain, which is ten times bigger than existing, frequently used datasets in this area. The E2E dataset poses new challenges (1) its human reference texts show more lexical richness and syntactic variation, including discourse phenomena; (2) generating from this set requires content selection. As such, learning from this dataset promises more natural, varied and less template-like system utterances. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact The E2E set was used in the E2E NLG Challenge we organized, which provides an extensive list of results achieved on this data. The interest in the E2E NLG shared task has by far outperformed our expectations. We received a total of 60 submissions by 16 institutions with about 1/3 of these submissions coming from industry. In comparison, the well established Conference in Machine Translation WMT'17 (running since 2006) got 31 institutions submitting to a total of 8 tasks. After the shared task was completed, the E2E dataset was made freely available online in full (including test data) and has been downloaded 41 times as of March 6, 2018. 
URL http://www.macs.hw.ac.uk/InteractionLab/E2E/
 
Description AdeptMind PhD overseas scholarship 
Organisation AdeptMind Inc
Country Canada 
Sector Private 
PI Contribution We are applying and extending our research in data-driven Natural Language Generation techniques to e-commerce. In particular, we are inverstigating how to translate search results into textual descriptions which support the user in decision making.
Collaborator Contribution AdeptMind funds this oversea PhD scholarship with a £88k cash contribution, as well as other in-kind contributions, such as invited research visits (incl. travel costs), student training (incl. summer schools), as well as sharing data sets and sponsoring data collection.
Impact This collaboration brings together research on natural language processing and advances machine learning, as well as expertise in e-commerce platforms.
Start Year 2017
 
Description Adobe Research 
Organisation Adobe Inc.
Country United States 
Sector Private 
PI Contribution One of my PhD students went for a summer internship to Adobe, where he continued working on a topic related to his PhD. When he came back, we refined the research hypothesis, ran some more experiments, and submitted a paper to ACL (= premier conference in the field). We already received the scores and we think it islikely that the paper will be accepted. This paper would be a very good candidate to submit to REF. My student used high performance computing from the HWU equipment account.
Collaborator Contribution Adobe supervised my students work for 3 months during the internship and also paid his flight and salary. Adobe then also sent us a research gift of £7000 and encouraged us to apply for further funding.
Impact TBA
Start Year 2019
 
Description Amazon Alexa Challenge 2017, 2018 
Organisation Amazon.com
Country United States 
Sector Private 
PI Contribution My team was selected to participate in the Amazon Alexa Challenge in two consecutive years: 2017 and 2018. The aim of this challenge is to build a social chat bot that can converse coherently and engagingly with humans on popular topics for 20 minutes. For the 2017 round, we were one of 12 teams selected out of a pool of over 100 applicants. For the 2018 round, we were 1 in eight teams selected out of ca. 200 applicants.
Collaborator Contribution We received a generous gift of $100,000 (2017) and $250,000 (2018) and various in-kind contributions worth ca. $100k for both years, e.g. free training and access to Amazon Web services, Alexa-enabled devices, weekly class with one of Amazon senior researchers, invited research visits to Amazon HQ in Seattle (including sponsored travel for the team) etc. We won 3rd prize for the 2017 challenge, which included a $50,000 cash prize for the students.
Impact Increased recognition and visibility of my research group and department.
Start Year 2016
 
Description EmoTech North Industry Knowledge Exchange 
Organisation EmoTech Ltd
Country United Kingdom 
Sector Private 
PI Contribution we collaborate on designing and implementing a conversational interface for Olly the Robot - a product developed by Emotech Ltd, an in-home robot with conversational capabilities. The Olly robot recently won 4 awards for Innovation at the CES showcase. (The CES Innovation Awards is an annual competition honoring outstanding design and engineering in consumer technology products over the world.) Recently showcased at CES '17 http://www.bbc.com/news/technology-38504512 The project outcome will directly contribute the Olly product of Emotech. Emotech will release 1000-1500 units in June/July via a Kickstarter program to gauge early adopter feedback. Full commercial release is expected in Q3/4 2017 at a retail price of $600-800 per unit. The revenue of Emotech LTD in 2017 is estimated to be £2m, and is expected to grow to £20-40m in 2018. Emotech North Ltd will be a NLP(Natural Language Processing) hub for Emotech. Its growth will create more employment positions, more collaborations with other industry partners and universities in Scotland.
Collaborator Contribution Cash contribution of £58k to support RA. Invited research visit to London (1 week) fully supported.
Impact Robotics hardware, neuroscience, human-computer interaction
Start Year 2016
 
Description IDS Oil and Gas 
Organisation Independent Data Services UK
Country United Kingdom 
Sector Private 
PI Contribution IDS has been providing operational reporting solutions to the upstream oil and gas industry for over twenty years. This project provides consultancy how to best use NLP and machine learning methods to mine information available in Oil & Gas data sets.
Collaborator Contribution IDS has send one of their employees to work at Heriot-Watt for a week. Also, a cash contribution of £5k was made.
Impact This project combines Data Science with NLP and applies it to Oil & Gas. The outcomes of this collaboration will be directly used in IDS services.
Start Year 2018
 
Title E2E NLG Challenge Metrics 
Description The software is an implementation of 5 respected automatic word-overlap-based metrics for scoring natural language (NLG) outputs. The metrics scripts were previously available elsewhere, but this piece of software unifies the interface and simplifies the usage of all 5 metrics, thus providing a more detailed evaluation of NLG outputs. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact The software was used for evaluation in the E2E NLG challenge (https://github.com/tuetschek/e2e-metrics) as the official automatic evaluation procedure. It was used by the organizers for the final evaluation as well as by most of the 17 participants submitting NLG systems to the challenge during their system development. 
URL http://www.macs.hw.ac.uk/InteractionLab/E2E/
 
Company Name ALANA AI LIMITED 
Description Alana AI is a spin-out of Heriot-Watt with an office in London. It provides products and services to build and enhance conversational AI. 
Year Established 2019 
Impact We are currently investigating several potential use cases with the Royal Blind and Education providers.
Website https://alanaai.com/
 
Description A presentation at the Pint of Science Festival (Edinburgh) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Ondrej Dusek gave a talk on smart speakers and conversational assistants at the Pint of Science Festival event titled "Language: From the Brain to Machines" in Edinburgh on 15 May 2018. The event also included a talk by Dr. Adam Lopez from University of Edinburgh on machine translation. It was attended by about 80 members of public and students, who seemed interested in the subject and posed questions during and after the talks.
Year(s) Of Engagement Activity 2018
URL https://pintofscience.co.uk/event/language-from-the-brain-to-machines
 
Description Amazon Alexa Summit 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited 3-day visit and presentation at Amazon HQ in Seattle to take part in Alexa Summit/ Symposium directed to industry practitioners and postgrad students.
Year(s) Of Engagement Activity 2017
 
Description BBC The Joy of AI 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact My research was featured in the BBC's documentary "The Joy of AI"
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/programmes/p06jt7j4
 
Description BBC interview 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview for BBC technology news
Year(s) Of Engagement Activity 2020
URL https://www.bbc.co.uk/news/technology-51064369
 
Description Conversational Agents course (HWU) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Contributed to the course 'Conversational Agents', attended by approximately 18 undergraduate students. The contribution included in particular the supervision of 5 of the students carrying out a project as part of their coursework. The project involved working with the dialogue system code developed so far in the Madrigal project and adapting it to a new domain (domain adaptation being one of the key aspects of Madrigal). This activity has enabled knowledge transfer to students and given them an opportunity to get both theoretical and practical experience with developing dialogue systems. Moreover, the interaction with the students and their feedback has triggered major improvements and further development of the Madrigal software.
Year(s) Of Engagement Activity 2017
URL https://sites.google.com/site/olemon/conversational-agents
 
Description DATAIA invited talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact I am invited to the DATAIA Institute, the French institute on AI to give a talk
Year(s) Of Engagement Activity 2020
URL http://dataia.eu/en/events/dataia-seminar-how-machines-learn-talk-challenges-and-opportunities-neura...
 
Description Diversity and inclusion in academic ICT research 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact I am taking part in the focus group Diversity and inclusion in academic ICT research run by the EPSRC and organised by Edinburgh Napier University.
Year(s) Of Engagement Activity 2017
URL https://www.epsrc.ac.uk/newsevents/news/ictdiversityinclusionresearch/
 
Description ECML-PKDD conference (Riva del Garda, Italy) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Attended the ECML-PKDD 2016 conference and gave a talk in one of its workshops, introducing the Madrigal project to peer researchers from both academia and industry. This conference visit also enabled many discussions with specialists in the field of machine learning, which was the topic of the overall conference, and an important aspect of the Madrigal project. The trip also included discussion with project collaborator Prof. Harry Bunt, who gave an invited talk at the workshop. The discussion included the planning of mutual research visits in 2017.
Year(s) Of Engagement Activity 2016
URL http://www.ecmlpkdd2016.org
 
Description Edinburgh Science Festival talk + discussion 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Ondrej Dusek gave a talk about the latest developments in conversational assistants (i.e., spoken dialogue systems) at the Edinburgh Science Festival event called "Your Robot Roommate", which concentrated on human-robot interactions. The event included talks by two other researches (Kerstin Dautenhahn, Boris Mocialov) and a panel discussion about the future of robotics. The event was attended by about 30 members of public, who were very interested in the subject and posed a lot of questions during the panel discussion and immediately after the event.
Year(s) Of Engagement Activity 2017
URL https://www.sciencefestival.co.uk/event-details/your-robot-roommate
 
Description Glasgow Psychology Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Invited seminar talk at the University's of Glasgow Institute of Neuroscience & Psychology. Cross-disciplinary engagement.
Year(s) Of Engagement Activity 2019
 
Description Interview for international news (WDR) 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview for German national radio - almost whole feature around our group and our research.
Year(s) Of Engagement Activity 2018
URL https://www1.wdr.de/mediathek/audio/wdr3/wdr3-kulturfeature/audio-sprich-mit-mir---versuche-mit-masc...
 
Description Interview for national news (Telegraph) 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interview for the Telegraph about Women in AI
Year(s) Of Engagement Activity 2019
URL https://www.telegraph.co.uk/technology/2019/03/08/artificial-intelligence-has-gender-problem-meet-pi...
 
Description Invited industry talk at Thomson Reuters 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Verena Rieser was invited to present her research to Thomson Reuters via an online seminar. This seminar will be broadcasted to all research employees of Thomson Reuters worldwide.
Year(s) Of Engagement Activity 2017
 
Description Invited keynote talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited keynote talk at Swisstext, an annual conference bringing businesses and academic practitioners together.
Year(s) Of Engagement Activity 2017
URL https://www.swisstext.org/2017/index.html
 
Description Invited seminar talk at University of Edinburgh 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Ondrej Dusek gave a seminar talk at the Institute of Language, Cognition and Computation, University of Edinburgh about the evaluation of natural language generation systems in the E2E NLG Challenge and about the Heriot-Watt University chatbot competing the 2018 Amazon Alexa Prize contest. The talk was attended by about 30 faculty members and students, who posed questions and commented on specific issues afterwards.
Year(s) Of Engagement Activity 2018
URL http://web.inf.ed.ac.uk/ilcc/news-events/previous-speakers/ondrej-dusek-seminar
 
Description Invited seminar talk at University of Glasgow 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact David Howcroft presented "Crowdsourcing and evaluating text quality" at the Information Retrieval seminar for University of Glasgow Computer Science. The presentation covered tools, platforms, and best practices for crowdsourcing studies in computational linguistics and highlighted new work on evaluation in Dr. Rieser's group. Faculty and students posed engaging questions afterward, with follow-on discussions lasting more than an hour.
Year(s) Of Engagement Activity 2020
URL https://samoa.dcs.gla.ac.uk/events/viewtalk.jsp?id=16744
 
Description Invited seminar talk at the University of Pennsylvania, US. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Verena Rieser gave an invited seminar talk at the University of Pennsylvania on: "From Dialogue Systems to Social Chatbots: Reinforcement Learning, Seq2Seq, and back again"
Year(s) Of Engagement Activity 2017
URL https://pricelab.sas.upenn.edu/clunch16-17
 
Description Media Interviews regarding the Amazon Alexa Challenge 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact Interview at Scottish TV "Live at 5" about the Amazon Alexa Challenge. Several related newspaper articles, e.g. Sunday Herald, the Scotsman, etc.
Year(s) Of Engagement Activity 2017
URL https://www.youtube.com/watch?v=ERSMhGFBxvw
 
Description Motion in Scottish Parliament recognising Amazon Alexa Prize 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact Motion S5M-09326: Parliament congratulates Heriot-Watt University on its success in the Amazon Alexa Prize
Year(s) Of Engagement Activity 2017
URL http://www.parliament.scot/parliamentarybusiness/28877.aspx?SearchType=Advance&ReferenceNumbers=S5M-...
 
Description NESTA 12 Women shaping AI 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact My profile was featured as one of 12 Women Shaping AI by NESTA.
Year(s) Of Engagement Activity 2019
URL https://www.nesta.org.uk/feature/12-women-ai/
 
Description NESTA interview - 12 women shaping AI 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Media interview and article published by NESTA (global innovation foundation)
Year(s) Of Engagement Activity 2019
URL https://www.nesta.org.uk/feature/12-women-ai/
 
Description Native Scientist German School Outreach 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Verena Rieser engaged school children in her research. The half-day event was organised by Alleman Fun (German Saturday School) and Native Scientist. The engagement activity was held in German.
Year(s) Of Engagement Activity 2016
URL http://www.macs.hw.ac.uk/RoboticsLab/news/german-native-scientist-volunteers-reaching-out-to-childre...
 
Description Panel member at CogX 2017, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact CogX was a two day event specifically focused on the impact that AI has across industry, government and society as a whole. The event included a Trade Expo, two days of discussions, debates and breakout sessions, as well as the inaugural CogX Awards. It attracted over 1,500 attendees to explore the most topical AI trends, the first and second order effects of the AI revolution, and the challenges, opportunities and recommendations for how to navigate the new landscape which is rapidly reshaping the world around us.
Year(s) Of Engagement Activity 2017
URL https://cogx.co/cogx2017/speakers/
 
Description Plenary keynote at 1st workshop on NLP for Conversational AI (ACL2019, Florence) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited plenary keynote at 1st workshop on NLP for Conversational AI (ACL2019, Florence)
Year(s) Of Engagement Activity 2019
URL https://sites.google.com/view/nlp4convai/program?authuser=0
 
Description Plenary keynote at 2nd workshop on Vocal Interactivity in-and-between Humans, Animals and Robots (VIHAR-2019) (Turing Institute, London) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited plenary keynote at 2nd workshop on Vocal Interactivity in-and-between Humans, Animals and Robots (VIHAR-2019) (Turing Institute, London)
Year(s) Of Engagement Activity 2019
URL http://vihar-2019.vihar.org/keynotes/
 
Description Plenary keynote at IVA 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited plenary keynote at 19th International Conference on Intelligent Virtual Agents (IVA 2019, Paris)
Year(s) Of Engagement Activity 2019
URL https://iva2019.sciencesconf.org/
 
Description Poster presentation at EurNLP: "Arguing for consistency in the human evaluation of natural language generation systems" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact David Howcroft presented "Arguing for consistency in the human evaluation of natural language generation systems" at EurNLP 2019, hosted at Facebook London. This presentation has fostered connections with researchers in the UK and abroad who are planning to collaborate on improving evaluation for NLG.
Year(s) Of Engagement Activity 2019
URL https://www.eurnlp.org/eurnlp-2019/accepted-abstracts
 
Description Poster presentation at EurNLP: Noise and Neural Natural Language Generation: Rubbish in, Rubbish out? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Karin Sevegnani presented "Noise and Neural Natural Language Generation: Rubbish in, Rubbish out?" at EurNLP 2019, hosted at Facebook London. In addition to presenting our findings to other researchers, this was Ms. Sevegnani's first presentation at an international research conference, contributing to her professional development.
Year(s) Of Engagement Activity 2019
URL https://www.eurnlp.org/eurnlp-2019/accepted-abstracts
 
Description Scottish Minister for Higher Education visit 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact Shirley-Anne Somerville, Minister for Higher Education, Further Education and Science, travelled to the Edinburgh Campus to meet with members of the University's 'What's Up Bot' team who recently returned from Amazon's prestigious AI competition, the Alexa Challenge.
Year(s) Of Engagement Activity 2017
URL https://www.hw.ac.uk/about/news/minister-given-insight-into-the-future-of-ai.htm
 
Description Short Interview for Herald Scotland 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Ondrej Dusek gave a short e-mail interview to Herald Scotland regarding the future of human-robot interaction and was quoted in an article "Tommorrow's World".
Year(s) Of Engagement Activity 2017
URL http://www.heraldscotland.com/life_style/15166452.Tomorrow__39_s_World/
 
Description SigDial panel discussion 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited Panel Member at 18th Annual SIGdial Meeting on Discourse and Dialogue Conference discussing "Natural Language Generation for Spoken Dialogue Systems". Other panel members were from Google (Head of NLG), Cambridge University and Bloomberg.
Year(s) Of Engagement Activity 2017
URL https://www.superlectures.com/sigdial2017/panel-discussion
 
Description Telegraph Pioneering Women in AI 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact My profile was featured in the Telegraph as a Pioneering Women in AI
Year(s) Of Engagement Activity 2019
URL https://www.telegraph.co.uk/technology/2019/03/08/artificial-intelligence-has-gender-problem-meet-pi...
 
Description Top 30 people to follow on Twitter 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact I was nominated as on in 30 top European people in AI to follow on Twitter.
Year(s) Of Engagement Activity 2019
URL https://sifted.eu/articles/30-ai-people-in-europe-to-follow-on-twitter/
 
Description WDR3 German radio interview 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Verena Rieser, Ondrej Dusek and other members of the Interaction Lab gave detailed interviews about current research in chatbots and dialogue systems to reporter Tom Schimmeck from the German WDR3 radio station. This was then featured in a 1-hour radio documentary titled "Sprich mit mir - Versuche mit Maschinen ins Gespräch zu kommen" (Talk to me - attempts at starting a conversation with machines), which aired on WDR3 on 29 December 2018.
Year(s) Of Engagement Activity 2018
URL https://www1.wdr.de/mediathek/audio/wdr3/wdr3-kulturfeature/audio-sprich-mit-mir---versuche-mit-masc...
 
Description Women@CS 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Verena Rieser organises a local support group for female students studying Computer Science, inspired by the "Sisters Clubs" in American universities. The goal is to attract and retain female UG students to study CS.
Year(s) Of Engagement Activity 2016
 
Description top 30 women in AI: UK Edition 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact I was nominated as #5 of top Women in AI in the UK by Re:Work
Year(s) Of Engagement Activity 2019
URL https://blog.re-work.co/top-30-women-in-ai-uk-edition/