📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

Efficient Cross-Domain DSL Development for Exascale

Lead Research Organisation: Imperial College London
Department Name: Earth Science and Engineering

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description Significant Achievements and Insights:

1. Successful Integration of MPI into MLIR: A pivotal accomplishment was the integration of MPI support into MLIR through a newly developed MPI dialect. We expect this to greatly impact the use of MLIR for distributed memory parallelism, marking a significant leap forward in compiler technology for high-performance computing (HPC). The acceptance of this dialect into the MLIR codebase highlights not only a technical breakthrough for DSLs in HPC, but it will also directly enable scalable AI and exascale computing applications.

2. Innovations with the xDSL Framework: Significant progress in developing xDSL as an alternative compiler layer for Devito represents a strategic move towards establishing a more generalized and efficient framework for DSLs targeting exascale applications. Although not yet finalized, the framework underlines the project's dedication to redefining DSL compiler design and paves the way for future advancements in this domain.

3. Advancements in Compiler Technology Applications: Devito has explored groundbreaking directions in compiler technology, developing an alternative code generation path that generates code for matrix-free callback functions that can be used in scalable solver libraries such as PETSc. This greatly widens the range of applications that Devito can be applied to - i.e., applications that require scalable implicit solvers, such as fluid flow optimization problems in renewable energies.

Meeting the Objectives:

The project's outcomes signify a considerable stride towards the envisioned goals, especially in DSL compiler design and the integration of high-level abstractions for exascale computing. The achievements in MPI support for MLIR and the ongoing development of xDSL underscore a robust commitment to overcoming the complexities of exascale DSL development, despite encountering hurdles in fully realizing all project ambitions. The Devito DSL has also also demonstrated that performance-portability is possible by extending it's support for AMD, Intel and Nvidia GPUs.

Implications for Future Research and Adoption:

The insights and advancements from this project lay a solid foundation for future explorations in DSL compiler technology and its applications in HPC and AI. Potential pathways for adoption and further research include:

- Academic and Research Institutions: could leverage these innovations for cutting-edge research in scalable computing and compiler technology.
- Industry Stakeholders: in data science, AI, and HPC may find valuable applications of the project's developments in enhancing software performance and scalability.
- Open-source Communities stand to benefit significantly from this project's collaborative and innovative spirit, potentially spearheading further advancements in compiler technology and DSL development. Today, Devito has nearly 900 people on its Slack channel, including a wide range of users from academia and industry.

In summary, this project has illuminated critical achievements and lessons in the realm of DSL compiler technology, contributing significantly to the field and identifying challenges and opportunities for future endeavors. The integration of MPI support into MLIR, alongside the development of xDSL and novel compiler technology applications, represents key milestones. Equally important is the recognition of areas needing further exploration, particularly in refining the xDSL framework to meet production standards, thus setting the stage for continued advancements in exascale computing and AI.
Exploitation Route The outcomes of this project present a myriad of opportunities for both academic and non-academic entities to advance the fields of high-performance computing (HPC), artificial intelligence (AI), and domain-specific language (DSL) development. For instance, integrating the MPI dialect into MLIR has significant implications for reducing the complexity and costs associated with deploying scalable AI and exascale computing solutions. This development could be a cornerstone for HPC and AI researchers looking to leverage distributed memory parallelism more efficiently.

In the academic realm, institutions can build upon the project's advances to further explore the intersection of compiler technology and DSLs, particularly in devising new languages or frameworks that enable domain scientists to harness the power of HPC without necessitating deep technical knowledge of the underlying hardware. This could democratize access to exascale computing resources across various scientific domains, fostering innovation and accelerating discovery.

Devito's growing adoption for the non-academic sector, specifically within the energy industry, underscores its potential to facilitate performance portability across different hardware platforms. This ensures that industries are not confined to a single hardware provider due to their choice of programming model, such as CUDA for Nvidia GPUs. By adopting Devito and the principles laid out in this project, companies can achieve greater flexibility in their computational strategies, optimizing for both cost and performance. Additionally, the project's contributions to xDSL/MLIR threads highlight the feasibility of creating new DSLs tailored to domain experts' specific needs, enabling them to develop solutions on HPC platforms efficiently.

Ultimately, the project's outcomes are poised to be leveraged by a broad spectrum of stakeholders, ranging from academic researchers and industry practitioners to members of open-source communities. The collaborative and innovative spirit demonstrated by the Devito project, evidenced by its active and growing Slack channel community (currently 900 members), will likely inspire further advancements and applications of DSL technology in various sectors.
Sectors Energy

Healthcare
 
Description The funding and research behind Devito and its subsequent evolution into Devito Codes, along with the commercialization through Devito Pro, have catalyzed a significant impact across economic, societal, and academic domains. Education: Devito has now been adopted to teach seismic imaging at Imperial College London undergraduate and postgraduate levels. Economic and Societal Impact: - Global Economic Performance: Devito Codes and its commercial counterpart, Devito Pro, have enhanced the economic competitiveness of the United Kingdom and beyond. By offering both an open-source platform and a proprietary extension with Devito Pro, Devito Codes has facilitated the adoption of high-performance computational methods across industries, notably in exploration geophysics. This dual approach has fostered innovation and provided cost-effective solutions for complex computational problems, potentially saving millions in development costs for companies. - Quality of Life and Health: Devito's extension into medical imaging projects signifies its contribution beyond the energy sector, impacting health and quality of life. By enabling high-performance simulations in medical imaging, Devito supports advancements in diagnosis and treatment, contributing to better health outcomes and advancing medical research. * https://www.stride.codes/* https://agencyenterprise.github.io/neurotechdevkit/ Academic Impact: Nucleation of New Research Areas: Devito's development as a high-performance software for seismic imaging and its growth into a comprehensive framework for simulation, inversion, and optimization have opened new research avenues. It has established a benchmark in using domain-specific languages and code-generation frameworks in scientific computing, inspiring further research in these areas. - Breakthrough in Computational Efficiency: By automating the generation of optimized parallel code for a variety of architectures, Devito has addressed a fundamental challenge in computational science: reducing the time and cost of developing high-performance computational codes. This breakthrough supports a wide range of scientific investigations, from geophysical exploration to medical imaging, and represents a significant leap in computational efficiency. Wider Impact Through Devito Codes: - Enhancement of Public Services and Policy: Through its applications in geophysics and medical imaging, Devito Codes contributes to the effectiveness of public services by providing tools for environmental monitoring, resource exploration, and healthcare. These applications have the potential to inform public policy, especially in areas related to energy exploration and public health. - Creative Output: Devito Codes supports the creative output of scientists and engineers by simplifying the process of developing high-performance computational models. This enables innovative approaches to solving complex problems and encourages creativity in computational methods and applications. - Training and Increased Research Capability: Devito Codes Ltd.'s focus on providing professional services, training, and bespoke software development has enhanced research capabilities by equipping researchers and professionals with specialized skills in high-performance computing and DSLs. This effort has built a knowledgeable community around Devito, as evidenced by its active Slack channel and broad user base, further contributing to the project's impact. In conclusion, the outcomes of this funding have been utilized in ways that significantly contribute to economic competitiveness, societal well-being, and the advancement of academic research. Devito's journey from a research project to a commercial product exemplifies the powerful impact of bridging academic innovation with industry application, paving the way for further advancements in computational science and its applications across sectors.
First Year Of Impact 2020
Sector Education,Energy,Environment,Healthcare
Impact Types Societal

Economic

Policy & public services
 
Title devitocodes/devito: v4.8.0 
Description Changes misc Updated contact information in the Code of Conduct. @ggorman (#2038) API symbolics: use devito floor instead of Undefined Function @mloubout (#2052) api: Add dimension-wise summing builtin and tests @mloubout (#1989) Examples Examples: Add Darcy flow example @sashaowen (#1998) examples: Add shallow water equations notebook @AtilaSaraiva (#1867) examples: invoke tti example with --constant argument @ofmla (#1914) Documentation docs: Update compiler summary image @georgebisbas (#2037) misc: Update docker/README.md @FabioLuporini (#1972) Update FAQ.md @FabioLuporini (#2010) misc: Add FAQ page (lifted from the wiki) @FabioLuporini (#1976) Compiler compiler: Patch data dependencies across Jumps @FabioLuporini (#2065) compiler: Implement graceful lowering of derivatives (aka "unexpansion") @FabioLuporini (#2060) compiler: Switch from aomp to clang for amd @mloubout (#2058) compiler: Extensions for parlang backends @FabioLuporini (#2042) compiler: Introduce int32 mode @FabioLuporini (#2041) compiler: Support shared memory in parlang backends @FabioLuporini (#2025) compiler: support for HPE Cray Clang compiler @georgebisbas (#2029) compiler: Better blocking heuristics and revamped linearization @FabioLuporini (#2020) compiler: Further misc improvements @FabioLuporini (#2012) compiler: Misc refactorings towards serialization support @FabioLuporini (#2009) Misc code generation improvements @FabioLuporini (#2001) compiler: Misc code generation fixes @FabioLuporini (#1994) compiler: Misc tweaks for backend-portable code generation @FabioLuporini (#1984) MPI return slice(0,-1) for glb_slices if glb_numb empty on an mpi rank @deckerla (#2004) mpi: Fix data distribution bugs [part 2] @rhodrin (#1949) compiler: Fix MPI mode diag2 does not need a MPIRegion @mmohrhard (#1992) Architectures and JIT arch: Add gcc 12 into legal configurations @ziyiyin97 (#2027) Bug Fixes compiler: Add cluster guard to AliasKey for safety @mloubout (#2045) dependencies: sympy 1.11 compatibility @mloubout (#2005) ci: Add Arm skip option to tests @mloubout (#2035) compiler: Ensure order invariance of candidates in ReducerMap.unique @ccuetom (#2033) compiler: Fix subdim argument mismatch @mloubout (#2019) symbolics: Fix absolute value warning for integer input @mloubout (#2018) types: Minor fixes to sparse function indices and implicit dims @mloubout (#2011) compiler: Patch placement of ConditionalDimension with multi-Dimension conditions @FabioLuporini (#2008) compiler: Prevent reordering of existing temps in CSE @mloubout (#2002) return slice(0,-1) for glb_slices if glb_numb empty on an mpi rank @deckerla (#2004) mpi: Fix data distribution bugs [part 2] @rhodrin (#1949) dsl: Patched cross-derivative fd_order bug @EdCaunt (#1988) compiler: Check jit_dir existence when saving @GlassOfWhiskey (#1983) Testing ci: Remove docker pruning from pytest-gpu @FabioLuporini (#2013) Continuous Integration docker: add AMD HIP build to base docker @mloubout (#2055) CI: switch macos runner to latest gcc version @mloubout (#2046) ci: improve accuracy of codecov. @ggorman (#2040) ci: Streamlining @ggorman (#2028) ci: Remove dangling docker layers @FabioLuporini (#2017) Installation pip prod(deps): update distributed requirement from <2023.2 to <2023.3 @dependabot (#2063) compiler: Switch from aomp to clang for amd @mloubout (#2058) docker: Switch to rocm 4.5.2 @mloubout (#2057) docker: add AMD HIP build to base docker @mloubout (#2055) pip prod(deps): update distributed requirement from <2022.13 to <2023.2 @dependabot (#2048) dependencies: sympy 1.11 compatibility @mloubout (#2005) pip prod(deps): update distributed requirement from <2022.12 to <2022.13 @dependabot (#2039) pip prod(deps): update distributed requirement from <2022.11 to <2022.12 @dependabot (#2031) pip prod(deps): update py-cpuinfo requirement from <=8 to <10 @dependabot (#2026) misc: Add packages to Docker base images @FabioLuporini (#2030) pip prod(deps): update distributed requirement from <2022.10 to <2022.11 @dependabot (#2021) pip prod(deps): update distributed requirement from <2022.9 to <2022.10 @dependabot (#1996) pip prod(deps): update distributed requirement from <2022.8 to <2022.9 @dependabot (#1987) 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact Over 700 members of our open slack channel. Wide adoption in the energy industry. Starting to be used in several medical imaging projects. 
URL https://zenodo.org/record/7639207
 
Title devitocodes/devito: v4.8.10 
Description Changes Dependencies deps: Fix sympy install for pypi API compiler: Drop SubDomainSet earlier during compilation for more graceful lowering @EdCaunt (#2393) Compiler compiler: Drop SubDomainSet earlier during compilation for more graceful lowering @EdCaunt (#2393) 
Type Of Technology Software 
Year Produced 2024 
Impact Devito is an open-source Python package designed for the implementation of optimized stencil computations, particularly finite-difference methods for solving partial differential equations (PDEs). By leveraging symbolic computation through SymPy, Devito enables users to define complex mathematical problems in a high-level, human-readable syntax, which is then transformed into highly optimized, architecture-specific code through automated code generation and just-in-time compilation. This approach significantly reduces development time and enhances computational performance across various platforms, including CPUs, GPUs, and distributed systems. Notable Impacts of Devito: 1. Advancements in Geophysical Imaging: - Subsurface Imaging: Devito has been instrumental in the energy industry and geophysical research communities for subsurface imaging applications. Its ability to efficiently solve wave equations has facilitated more accurate and faster imaging of the Earth's subsurface structures, aiding in resource exploration and geohazard assessment. - Elastic Modeling and Anisotropic Inversion: Recent developments using DevitoPRO have enabled large-scale elastic modeling and anisotropic inversion, allowing for more detailed and realistic representations of subsurface properties. These advancements are crucial for improving the accuracy of seismic imaging and interpretation. 2. Integration with Machine Learning: - SLIM Group at Georgia Tech: The Seismic Laboratory for Imaging and Modeling (SLIM) at Georgia Tech has integrated Devito into their Julia-based framework, JUDI, to enhance seismic imaging and machine learning applications. This integration has led to cutting-edge developments in probabilistic full-waveform inversion and serverless cloud imaging, bridging the gap between academic research and production-level seismic applications. - Learned Multiphysics Inversion: Devito has been utilized in frameworks combining wave-equation-based inversion with machine learning techniques. This integration facilitates the development of learned neural surrogates for multiphase flow simulations, advancing methodologies in computational geophysics and related fields. 3. Commercialization and Industry Adoption: - Devito Codes Ltd: Beyond its open-source roots, Devito has been commercialized through Devito Codes Ltd, offering enterprise-level solutions and support. This commercialization has expanded Devito's reach and facilitated its adoption in various industrial applications. - Industry Collaborations: Devito has been integrated into several proprietary software solutions across different sectors: - COFII by Chevron: Utilizes DevitoPRO for advanced seismic imaging and inversion processes. - DUG Wave by DownUnder GeoSolutions: Employs the open-source version of Devito for seismic data processing and imaging. - Imaging AnyWare by TGS: Incorporates DevitoPRO to enhance their subsurface imaging capabilities. - Neurotech Development Kit (NDK): Uses open-source Devito for developing neural technologies. - Stride: An open-source library for ultrasound modeling and tomography, built on top of Devito, providing flexibility and scalability for medical imaging applications. - Sonalis Imaging: A medical imaging startup leveraging DevitoPRO to revolutionize brain imaging using 3D ultrasound computed tomography. 4. Collaborations with Technology Leaders: - NVIDIA: Devito has been featured in NVIDIA's developer resources, highlighting its capabilities in designing highly optimized finite-difference kernels for inversion methods. - AMD: Collaborations with AMD have focused on optimizing Devito for AMD's GPU architectures, enhancing performance portability and computational efficiency. - Intel and Microsoft Azure: Devito has been optimized for Intel architectures and integrated into cloud computing platforms like Microsoft Azure, demonstrating its scalability and adaptability to various high-performance computing environments. Devito has significantly impacted computational science and engineering through these diverse applications and collaborations, providing a versatile and efficient tool for modeling, simulation, and data inversion across multiple disciplines. 
URL https://zenodo.org/doi/10.5281/zenodo.12728258
 
Description Devito industry workshop at Rice Energy HPC 2023 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Delivered training on using Devito to 40-50 industry participants. An expert panel from the industry to debate Devito roadmap. Spearheading the development of industry standards for benchmarking industry codes that will will at exascale.
Year(s) Of Engagement Activity 2023
URL https://www.energyhpc.rice.edu/schedule