Discovery: Pattern Discovery and Program Shaping for Manycore Systems

Computing pervades our lives, impacting our health, work, entertainment and social interaction. Over recent years, the technology inside the devices providing these services has undergone a radical change: where once, processing was undertaken by relatively homogeneous "sequential" devices, in which essentially one thing happened at a time, the new systems compose a range of specialized devices, some targeting specific problem sub domains, and almost all exhibiting considerable "parallelism", where many things can happen at the same time. This is true on all scales, from the internals of a mobile phone, to the massive data centres which serve web applications such as Google. This poses a substantial challenge for the software industry: writing correct and efficient programs for heterogeneous, highly parallel systems is much harder than for current technologies and most developers lack the skills and training to write safe and efficient code. Faced with this difficulty, software developers will often avoid writing parallel code completely, or else will use inappropriate, non-scalable and error-prone approaches based on explicit threads of program execution. Given the hardware trend towards increasingly complex, increasing parallel (manycore) systems, this is an inherently short-term strategy that is doomed to failure,

Our project addresses this issue. Our key insight is that humans in general, are very good at using patterns to understand, predict and act in the real world. This insight translates into the world of software engineering in general, and parallel heterogeneous programming in particular. Our work will help programmers to recognize patterns in pre-existing and new applications, and to transform these pattern occurrences into forms which allow them to be exploited, adapted and run effectively on the new hardware platforms. The systems we develop will work in partnership with software developers, reducing the complexity of the task, automating and semi-automating the development task. The result will help the industry to develop new applications, and to update existing applications, with less effort, fewer errors and better resilience as the underlying technology continues to evolve.

The impact of the Discovery project will be delivered through the economy, society and people.

The Discovery project will have both near-term and long-term economic impact. Discovery engages directly with major UK and international companies to deliver direct benefits in terms of the production of high-quality, scalable and future-proof software for emerging computer architectures. Software is an essential, but sometimes invisible, component of the modern, smart economy. Software development forms a global economic activity that is worth in excess of £116bn today, and which is growing year-on-year. It contributes in an essential way to a growing ICT industry that is worth in excess of £1,000bn. The results of the Discovery project will allow the software development industry to address global ICT needs in a more cost-effective and resource-efficient way. We will cooperate with major industry concerns, run workshops and tutorials to maximise uptake of the Discovery concepts and technologies, and adopt a dissemination and communication strategy that is designed to engage with industry as well as academia. This will include posters, booths and presentations at key industry conferences and events, such as the HiPEAC European network of excellence. Finally, we will engage with ongoing pre-commercialisation work funded by Scottish Enterprise, that will provide opportunities for direct commercialisation through e.g. the formation of a start-up company, commercial licensing of the Discovery technology etc.

In the very long-term, the ultimate impact of our research will be societal. The prevalence of ICT, seen and unseen, in modern life needs no further explanation. All of this technology is vulnerable to the software challenges that our work seeks to address. More specifically, our results will lead to reductions in the time and human resource required to develop software, and to improvements in the execution profile of the resulting applications, with a either reduced time and/or energy required, or higher quality results obtainable within a fixed resource budget, or a combination of these. In more concrete terms, this means savings in money, energy and time across the ICT spectrum, with the saved resource being redeployable to new societal goals, or to improved response to existing goals. As our "Pathways to Impact" document discusses in more detail, there will be further impacts along the way, notably within the software industry itself, where the development of our ideas to industrial strength tools offers significant benefit to the economies of the countries where this work takes place.

Finally, the project will have significant impact on people. It will provide opportunities for development of junior researchers as co-investigators, leading to new career opportunities and providing capacity increases in an area of strategic research importance. It will also link directly to the CDT in Pervasive Parallelism at Edinburgh University, providing opportunities to influence and benefit from PhD research, including generating exciting, challenging but achievable research projects. This will bring increased benefit and focus to the CDT programme as a whole, developing a new generation of research-informed people who will spread Discovery ideas and technologies throughout the UK and international research and high-technology sectors.