Efficient Cross-Domain DSL Development for Exascale
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Informatics
Abstract
Developing scientific software, for example for climate modeling or medical research, is a highly challenging task. Domain scientists are often deeply involved in low-level programming details just to make their code run sufficiently fast. These tedious, but important, optimization steps significantly reduce the productivity of scientists.
Domain specific languages (DSLs) revolutionize the productivity of domain scientists by enabling them to focus on scientific questions rather than making their code run fast. Sophisticated DSL compilers automatically generate high-performance code from domain-specific high-level problem descriptions.
While there are individual successes, the existing landscape of DSLs is scattered and the reuse of software components in DSL compiler implementations is limited as traditionally DSL compilers are built in isolation. This results in high development costs of new DSLs and prevents many DSLs from ever achieving a level of maturity and sustainability that enables uptake by the scientific community.
This project revolutionizes the design of DSL compiler implementations by leveraging the breadth and cross-industry support of the MLIR compiler and Python ecosystems. Python is the tool of choice for application developers in many domains, such as machine learning, data science, and - we believe - an important component of the future of High Performance Computing software. This project establishes MLIR as a common representation for code at multiple levels of abstraction in DSL compiler development. DSLs embedded in various host languages, including Python and Fortran, will be easily built on top of MLIR. Instead of building DSL compilers as isolated monolithic towers, our research will build a toolbox that enables developers to build DSLs using a rich ecosystem of shared intermediate representations IRs and optimizations.
This project evaluates, drives, and demonstrates the DSL design toolbox to build the next generation of DSLs for Seismic and Climate Modelling as well as Medical imaging. These will share common software components and make them available for other DSLs. An extensive evaluation will show the scalability of DSL software towards exascale.
Finally, this project investigates how future disruptors, including artificial intelligence, data science, and on-demand HPC-as-a-service, will shape and influence the next generations of high performance software. This project will work towards deeply integrating modern interactive data analytics and machine learning methods from the Python ecosystem with high-performance scientific code.
Domain specific languages (DSLs) revolutionize the productivity of domain scientists by enabling them to focus on scientific questions rather than making their code run fast. Sophisticated DSL compilers automatically generate high-performance code from domain-specific high-level problem descriptions.
While there are individual successes, the existing landscape of DSLs is scattered and the reuse of software components in DSL compiler implementations is limited as traditionally DSL compilers are built in isolation. This results in high development costs of new DSLs and prevents many DSLs from ever achieving a level of maturity and sustainability that enables uptake by the scientific community.
This project revolutionizes the design of DSL compiler implementations by leveraging the breadth and cross-industry support of the MLIR compiler and Python ecosystems. Python is the tool of choice for application developers in many domains, such as machine learning, data science, and - we believe - an important component of the future of High Performance Computing software. This project establishes MLIR as a common representation for code at multiple levels of abstraction in DSL compiler development. DSLs embedded in various host languages, including Python and Fortran, will be easily built on top of MLIR. Instead of building DSL compilers as isolated monolithic towers, our research will build a toolbox that enables developers to build DSLs using a rich ecosystem of shared intermediate representations IRs and optimizations.
This project evaluates, drives, and demonstrates the DSL design toolbox to build the next generation of DSLs for Seismic and Climate Modelling as well as Medical imaging. These will share common software components and make them available for other DSLs. An extensive evaluation will show the scalability of DSL software towards exascale.
Finally, this project investigates how future disruptors, including artificial intelligence, data science, and on-demand HPC-as-a-service, will shape and influence the next generations of high performance software. This project will work towards deeply integrating modern interactive data analytics and machine learning methods from the Python ecosystem with high-performance scientific code.
| Title | Project website and developer zone |
| Description | The main project website, which we view as a landing place for DSL developers to use as a hub for DSL ecosystem, documentation, tutorials etc. Whilst some of these activities will be further matured as the research progresses, crucially the framework and software has been set up such that we can easily add these components and keep them up to date as activities progress. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2021 |
| Impact | As a landing page we have already used it to generate interest and as a focal area for the project. |
| URL | https://xdsl.dev/ |
| Title | xdsl |
| Description | A modern SSA-based compiler implemented in Python |
| Type Of Technology | Webtool/Application |
| Year Produced | 2021 |
| Open Source License? | Yes |
| Impact | We taught a compiler lecture based on xdsl which resulted in many students picking up xDSL |
| URL | https://github.com/xdslproject/xdsl |
| Description | Birds of a Feather (BoF) session at SC21 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Birds of a feather session at SC21, which is the main world-wide supercomputing conference held in St. Louis USA November 2021. With a world-wide audience we split the hour session based on a short talk, three concurrent break-out sessions, and then a summary session. Idea was to explore consolidation of DSL ecosystems, as focused on the xDSL project, and start to build a global network around this. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://xdsl.dev/2021-11-10-bof-sc21/ |
| Description | DSL keynote at IWOMP |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Co-I of project gave keynote talk at International Workshop on OpenMP (IWOMP) conference about DSLs, describing ecosystem consolidation as per the xDSL project. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://xdsl.dev/2021-09-14-iwomp-keynote/ |