libhpc: Intelligent Component-based Development of HPC Applications

Libhpc aims build and demonstrate an environment for the component-based construction of HPC applications. This environment will enable method implementers to publish and share their contributions as multi-implementation adaptable components, shareable across domains and architectures, making them inherently more re-usable. End users will then be able to compose applications without reference to underlying machine or processor architectures. The aim of libhpc is to initiate a software development pattern for HPC that has knowledge capture and re-use at its core.Libhpc will be based on the combination of two long lines of research: that of Professor Sherwin in the development and application of spectral/hp element codes and that of Professor Darlington in the development of innovative application development methodologies. The project will take as its starting point the Nektar++ methods library and its application to the modelling of blood flow through an aortic arch. This application and the library will be re-factored to identify the necessary re-usable components and co-ordination forms required. These components will be designed and implemented, targeted at a variety of architectures, including, distributed memory clusters, shared memory clusters and GPU processors. A software methods repository with supporting meta-data will be constructed along with application-construction and mapping tools. This environment will then be used to re-express the aortic modelling application and map it systematically to a number of different architectures and machine configurations. These applications will be benchmarked to assess their performance and the improvement in human productivity and reusability gained.

If successfully carried out we firmly believe that this project could have considerable long-lasting impact addressing, as it does, fundamental issues concerning the way HPC applications are developed and deployed. As we have discussed earlier we consider that there is a great amount of knowledge concerning HPC methods and their implementations that is currently not captured in a manner that allows it to be effectively re-used or re-deployed. Our framework allows the expertise and knowledge of experts in HPC methods and their implementations to be captured explicitly in a manner that allows it to be systematically re-used. Our method is a classic example of the machine-tool paradigm - invest effort early in the development cycle to develop methods that are generic and re-usable and, thus, save much more effort downstream when these methods are used. The development framework would provide a much more efficient way of building and using HPC applications, supporting re-use and re-application of methods across a variety of applications and application types. The framework would also provide a pedagogical vehicle to help teach, in structured way, numerical methods, solvers, their relationships and implementation techniques, a key technology or discipline - numerical methods or computational engineering - that is currently lacking such a vehicle. The principal beneficiaries are: 1. Domain scientists who use computational tools. Today this includes almost every scientist. They will benefit by being able to design and use extremely flexible applications which can be run on diverse computing utilities with improved performance. 2. Numerical methods implementers. They will have access to a method for publishing and sharing their work, which is otherwise typically buried in machine-bound implementation code. 3. Utility computing operators. This includes both research grid operators and commercial operators such as Amazon Cloud. libhpc's flexible format will allow researchers to deploy to on-demand grid configurations, thus increasing uptake of grid resources and increasing ROI, especially in institutional settings. 4. Machine vendors. By exposing metadata about their products, vendors will encourage purchase and use. 5. Scientific research as whole. This will benefit by having access to improved methods. With a more systematic building block approach, it is likely that methods development will accelerate.