ReComp: sustained value extraction from analytics by recurring selective re-computation

As the cost of allocating computing resources to data-intensive tasks continues to decrease, large-scale data analytics becomes ever more affordable, continuously providing new insights from vast amounts of data. Increasingly, predictive models that encode knowledge from data are used to drive decisions in a broad range of areas, from science to public policy, to marketing and business strategy. The process of learning such actionable knowledge relies upon information assets, including the data itself, the know-how that is encoded in the analytical processes and algorithms, as well as any additional background and prior knowledge. Because these assets continuously change and evolve, models may become obsolete over time, leading to poor decisions in the future, unless they are periodically updated.

This project is concerned with the need and opportunities for selective recomputation of resource-intensive analytical workloads. The decision on how to respond to changes in these information assets requires striking a balance between the estimated cost of recomputing the model, and the expected benefits of doing so. In some cases, for instance when using predictive models to diagnose a patient's genetic disease, new medical knowledge may invalidate a large number of past cases. On the other hand, such changes in knowledge may be marginal or even irrelevant for some of the cases. It is therefore important to be able, firstly, to determine which past results may potentially benefit from recomputation, secondly, to determine whether it is technically possible to reproduce an old computation, and thirdly, when this is the case, to assess the costs and relative benefits associated with the recomputation.
The project investigates the hypothesis that, based on these determinations, and given a budget for allocating computing resources, it should be possible to accurately identify and prioritise analytical tasks that should be considered for recomputation.

Our approach considers three types of meta-knowledge that are associated with analytics tasks, namely (i) knowledge of the history of past results, that is, the provenance metadata that describes which assets were used in the computation, and how; (ii) knowledge of the technical reproducibility of the tasks; and (iii) cost/benefit estimation models.
Element (i) is required to determine which prior outcomes may potentially benefit from changes in information assets, while reproducibility analysis (ii) is required to determine whether an old analytical task is still functional and can actually be performed again, possibly with new components and on newer input data.

As the first two of these elements are independent of the data domain, we aim to develop a general framework that can then be instantiated with domain-specific models, namely for cost/benefit analysis, to provide decision support for prioritising and then carrying out resource-intensive recomputations over a broad range of analytics application domains.

Both (i) and (ii) entail technical challenges, as systematically collecting the provenance of complex analytical tasks, and ensuring their reproducibility, requires instrumentation of the data processing environments. We plan to experiment with workflows, a form of high level programming and middleware technology, to address both these problems.

To show the flexibility and generality of our framework, we will test and validate it on two, very different case studies where decision making is driven by analytical knowledge, namely in genetic diagnostics, and policy making for Smart Cities.

The ReComp software will benefit consumers of the knowledge outcomes from analytics, by ensuring that the value initially associated with those assets is retained over time. It will also benefit the organisations performing the analyses, by helping them manage their analytics budget through prioritisation of their re-computation efforts.
In the short term, our impact strategy will focus on the user groups associated to the Genomics and Urban Observatory case studies. This will give us iterative feedback on the ReComp tools and techniques, ensuring that they are both useful and useable. The software will be developed through a user-centric design model, whereby users and subject experts from the case studies are going to be engaged from the start and in each phase of development and release cycles.

Genomics: local experts are biomedical researchers at the IGM, who will provide both unsolved or inconclusive patient cases to the project, as well as validating the usability of the ReComp software. The main benefit for this group is the ability to systematically identify opportunities to address previously unsolved cases. This will translate into substantial savings, as IGM researchers estimate the success rate for diagnosis of rare diseases at about 25%, and typical processing fees figures per sample by commercial genetic diagnostics services are about £500. The engagement of IGM researchers is underpinned by the participation of their director - P. Chinnery - as CO-I.
Urban Observatory: the anticipated impact is to inform public policy by providing local government with precise and current knowledge of threats to public welfare, as well as of opportunities for optimisation of public services. P. James - leader of the Urban Observatory and CO-I on the project, will manage user engagement with the project.
As an additional benefit the Meta-K repository will provide a structured knowledge base for the growing volume of sensor data that the group is expected to generate for years to come.
Both these case studies offer a pathway to societal as well as economic benefit - improving health outcomes and city management will both have major benefits for the well-being of society.

In the medium and long-term, ReComp has the potential to benefit knowledge-driven companies and organisations that rely on analytics for decision-making. Impact to them is in terms of continued value from analytics knowledge over time, and financial savings through selective re-computation.
Promotion of ReComp uptake by these organisations will be pursued through non-academic dissemination and promotion channels, as explained in the Pathway document.

Academic impact will be achieved through publication and dissemination in the big data area, but also specifically in e-science venues including the annual IEEE International e-science conference, and the Future Generation Computer Journal, which are known to welcome contributions on data architectures in support of science.

ReComp will be released as open source to encourage adoption and customisation, and made available as a service on a cloud infrastructure. A sustainability plan will be formulated to facilitate this, possibly in collaboration with the Software Sustainability Institute (SSI). In addition to adopting a user-centric development model for the software deliverables, we will promote the framework both through demo events and academic dissemination, and by user training events.

We will also leverage active collaborations between the Digital Institute (DI) at Newcastle University, led by Prof. Watson (CO-I), and data science researchers in a wide range of disciplines, including e-Health, Internet of Things, and wearable computing.

The DI also actively engages with external organisations though its Cloud Innovation Centre. This gives direct routes for the outputs of the project to be widely deployed within and beyond Newcastle University, in companies as well as by research teams