ProvTemp: Provenance templates as a method for facilitating provenance capture and simulating provenance data

Our world is increasingly driven by data. Medical, economic and political decisions are made based on the results of automatically analysing ever-growing volumes of data. Whether these are patient treatment decisions or stock trading recommendations, if we are to trust the decisions being made, we need to have insight into the workings of these systems and achieve understanding of their outputs - referred to as their provenance.

Related to the issue of trust is the concept of reproducibility in scientific discovery, as the ultimate test of findings' validity. Science is now all but impossible without data-intensive infrastructures, but these changes make research harder to verify and follow using traditional "pen-and-paper" methods, and new techniques are required to ensure correctness. A number of recent studies looked into published research in certain areas, only to find that a minority could be reproduced using the information provided. Understanding the provenance of the data and processes that we are relying on has never been more critical.

Data provenance is a research field dedicated to standardised, uniform, representation of the network of data products, tasks that create and use those data, and the human and software actors who perform these tasks - typically represented as provenance graphs. Popular in "computational" disciplines that have long relied on scientific software, provenance is now becoming relevant and necessary to areas which have only recently become data-driven and which operate using multiple disjointed software tools.

In order to facilitate the adoption of provenance in these disciplines, ProvTemp project is modeling provenance templates - the provenance graph fragments that multiple software tools can compose into a unified, meaningful trace of conducted research. A set of templates is defined by the scientists, describing the research details that need to be captured, and these are then translated into concrete provenance data. This theoretical work has two immediate applications. The first is a method for introducing provenance into scientific environments by integrating with existing software tools, minimising the effort needed for the developers of those tools to start capturing provenance. Second is a mechanism for using the templates to simulate realistic provenance data that would be produced from those templates, allowing them to be tested to ensure they are sufficiently informative for the intended purpose, e.g. publishing details of research task, providing legally required audit trail etc.

The ProvTemp approach shall be evaluated on the example of modelling a clinical trial. The medical research community is a typical example of a non-computational discipline becoming increasingly data-driven, and it is currently moving towards big data enabled, intelligent infrastructures through use of data routinely captured in Electronic Health Record systems. The trend in medical research is towards Learning Health Systems, which seek to maximise and optimise the use and benefit of EHR data in clinical research and practice. The EU TRANSFoRm project, implemented a prototype software infrastructure for the Learning Health System, and conducted an international clinical trial, driven by EHR data. ProvTemp approach will replicate the trial execution using provenance templates, and examine the produced provenance data to ensure our method is valid and applicable to future clinical trials.

In addition to the clinical trial work, we shall work closely with UK's Software Sustainability Institute which promotes sustainable software technologies. SSI shall assist in ensuring that ProvTemp is generalisable and relevant to other scientific disciplines. We shall also engage the public in defining the wider questions around reproducibility and quality of research. Finally, ProvTemp will produce a roadmap for further research, taking stock of the work done and identifying future opportunities.

ProvTemp will facilitate adoption of provenance technologies in non-computational disciplines, such as medicine, that rely on heterogeneous software and lack the computational infrastructure the existing provenance solutions integrate with. By doing this, we shall move these disciplines closer to the vision of reproducible research, reduce administrative costs by automating the audit task, and support trust by increasing transparency of the research task.

These are issues that are of high importance to academics, industry and the public. Aside from an active data provenance research community in the UK, there is a fresh initiative around Research Objects, portable abstractions of a research task, encompassing source data, provenance, structured documentation, associated software snapshots and all other entities contributing to scientific findings. Industry, particularly Contract Research Organisations (CROs) conducting clinical trials and pharmaceutical companies, are under increased scrutiny to have readily available proofs that their data management procedures are effective and conformant to national and international legislation. With personal data on the Web, such as browsing habits and purchase histories, being routinely mined for commercial information, the public is rightfully concerned how will their private data be used in research and increasingly seeks assurance and understandability of the process before giving consent to use of their data.

We shall address public engagement through an active social media presence on blog, Twitter, and LinkedIn, complemented by series of public talks via Cafe Scientifique grassroots science movement, and the Pint of Science initiative. King's College London offers several venues for involving the public in science, which will be pursued, together with similar mechanisms at the Imperial partner.

Software Sustainability Institute, also a partner on the project, seeks to improve the quality of software engineering in science and sees provenance work as one of the key contributing factors to that. As part of their remit, SSI runs the Research Software Engineers community, with annual Collaborations Workshop, and a range of other workshops and activities. By promoting ProvTemp through these channels, we shall ensure that we reach a wide audience of research software developers who are key to popularising our approach.

The PI shall engage key UK academic institutions involved in provenance work through a series of academic visits and presentations at relevant workshops and conferences, including International Provenance Annotation Workshop (IPAW) and Theory and Practice of Provenance (TAPP). The clinical trial use case will also be presented in medical informatics journals and conferences, such American Medical Informatics Association (AMIA) Summits. Of particular use will be aligning our work with the wider Research Objects initiative for sharing and publishing academic research, and we shall investigate the impact of ProvTemp on applicability of Research Objects to non-computational disciplines, that have not been tackled yet.

At the end of the project, a roadmap will be produced, drawing on the insights gained through the engagements and with input from academics in relevant fields. This roadmap will articulate the open problems that need to be addressed to achieving reproducibility of data-producing tasks in scientific research and position these problems in relation to the results of ProvTemp. We will review the current state of the art that addresses these problems, identify the research communities contributing to these areas and provide examples of applications that either already do or could in the future benefit from provenance templates.