nlvis: Natural Language Interaction for Visual Data Analysis

The unprecedented increase in the amount, variety and value of data has been significantly transforming the way that scientific research is carried out and businesses operate. As data sources become increasingly diverse and complex, analysis approaches where the human and the computer operate in collaboration have proven to be an effective approach to derive actionable observations. This is achieved through an iterative human-computer dialogue where the knowledge and the creativity of the human meets the power of computation. In such human-in-the-loop data analysis approaches, interactive visualisation methods are core facilitators of this dialogue. However, these methods still rely on conventional, not often intuitive interaction mechanisms that can introduce unnecessary complexities into the process. There is an urgent need to rethink the ways how analysts interact with visualisations in data-intensive analysis situations. The recent advances in natural language based interaction methodologies offer promising avenues to address that.

This project aims to develop a fundamental understanding of how analysts can use natural language elements to perform visualisation empowered data analysis and use that understanding to develop a framework where natural language and visualisation based interactions operate in harmony. The project then aims to demonstrate how such a multi-modal interaction scheme can radically transform the analysts' experience with the goal of achieving significant improvements in the value and the volume of actionable observations generated.

Within the project, we will initially identify and develop a taxonomy of natural language interaction elements for describing visualisations and for carrying out a visual data analysis process. Here, we will inform our investigation with findings from data collected through crowd-based survey methodologies. We will then design a conceptual framework that facilitates an iterative data analysis process through interactions with both natural language and visualisation elements. We will make use of the data analysis and visualisation related language taxonomy from the earlier stage to define the scope and the capabilities of the interaction elements.

The project will then move on to realising its vision through a prototype where the conceptual framework will operate through the help of an established conversational interface mechanism. The prototype will involve a combination of natural language and visual interaction capabilities and will also incorporate underlying computational capacities. We will then evaluate our approaches through a series of carefully designed use-cases that encompass common visual analysis scenarios. Our success criteria will be to achieve enhanced engagement and improved productivity during the visual analysis of complex data-intensive problems.

Potential beneficiaries of the outputs of this project ranges widely from academic researchers, professional data analysts, data analysis industry, and the general public. For visualisation and visual analytics researchers, findings will benefit researchers who are working on understanding user-intent and mechanisms of sense-making in interactive visual analysis processes. For businesses that offer visualisation-empowered solutions to their customers (according to some reports, the visualisation market size is expected to reach a $2.8 Billion by 2020), the framework developed will provide the basis for new forms of products that are easier to learn and engage with. For professional data analysts, the novel interaction capability will offer a more fluid and natural experience, improving their efficiency and positively impacting the quality of observations. For the general public, natural interaction mechanisms will provide an enhanced experience when using data-intensive products that are becoming to be widely adopted.

The project plans two main pathways to impact and a number of side activities to widen the impact further. The two main activities are Knowledge Transfer through Project Partnership and a public engagement workshop. We also plan to organise more informal public engagement activities, and also try and increase the local outreach of the project. The PI will use the last 6 months of the project to focus on widening the impact of the project. The following are the main pathways to impact:

KNOWLEDGE AND TECHNOLOGY TRANSFER THROUGH PROJECT PARTNERSHIP: The first and most concrete pathway to impact plan of the project is through the project partner company named Redsift. They are looking into innovative ways to inform the development of their conversational analysis solution and data analytics capabilities within their product range. Therefore the impact of the research results will be almost immediate in this pathway to impact.

DISSEMINATION and ENGAGEMENT WORKSHOP: The second pathway to impact is through an engagement and dissemination activity that aims to get together experts from a wide range of academic, industrial and governmental institutions, and facilitate a lively discussion around the topic of "The role of the user/interaction in Data Science".

ACADEMIC PUBLICATIONS: During the duration of the project, we aim to produce a Short paper and a Workshop paper for the EuroVis 2018 conference to share the initial work-in-progress results. We then aim to produce a number of scientific papers: i) one detailing the observations from WP1 and the visualisation, task, and analytical intent related vocabulary ii) one describing the design process of the conceptual framework along with the results of the evaluation study. Even after the project has ended, we plan to work on position papers and other workshop papers that will discuss the implications of the utilisation of natural language approaches within visual analysis.

PUBLIC ENGAGEMENT: We aim to engage in activities where we can present project results to the general public. We will aim to take place in digital technology related activities that are organised in popular, social spaces. A couple of candidate events are: Digital Design Week at Victoria & Albert Museum, or appropriate themed events within Science Museum activities such as Science Museum Lates.

OPEN ACCESS TO PROJECT RESULTS: We will make sure that all the projects' outputs are available as open source. Any published paper will be made available through the Open Access System of CITY. And whenever there are no limitations we will ensure the anonymity of the data and make the data available along with the research papers through services such as GitHub.