BEM++ Stage 2

Lead Research Organisation: University College London
Department Name: Mathematics

Abstract

Many real-world problems in homogeneous media, such as electromagnetic scattering from an airplane or the noise radiation from a car, can be formulated as integral equations over the boundary of the domain. The advantage of such formulations is that only the boundary and not a (possibly infinite) medium needs to be discretised.
Computing solutions of such a boundary integral equation is the task of a boundary element method (BEM).

We have developed BEM++ (www.bempp.org), a modern open source boundary element software library, which can solve a wide range of problems in electrostatics, diffuse optics and acoustics. Support for electromagnetic and elasticity problems is coming soon. The library can be used either from C++ or via a convenient scripting interface directly from Python.

The current version of BEM++ already parallelises on modern multi-core workstations and has been deployed successfully for the computation of light diffusion in optical tomography applications. In this Stage 2 proposal we build on the successful core library and extend it to solve challenging application problems on modern heterogeneous computing architectures. To achieve this goal the following software library components will be developed as part of the project.

1.) We will implement hierarchical matrix and fast multipole methods on distributed CPU/GPU clusters. These methods make possible the fast solution of boundary integral equation problems with millions of unknowns and are essential for very large application problems.

2.) We will interface BEM++ with Dune, a well known high-performance library for large scale grid based applications such as finite element and finite volume methods. While boundary element formulations are suitable for homogeneous media problems they do not work for heterogeneous media, where finite element methods (FEM) are frequently being used. For coupled problems, where parts of the medium are homogeneous and others are heterogeneous it is often beneficial to couple FEM and BEM methods. The interface to Dune will allow us to do this, and to solve complicated coupled problems on large parallel computing architectures.

These extensions will allow us to solve large scale real world application problems. The first medical application that we will study is High Intensity focused ultrasound (HIFU) treatment. The principle here is to use focused ultrasound targeted at a tumor to destroy the cancerous tissue by localized thermal expansion. However, to properly plan HIFU treatment, careful numerical simulations are necessary and we will build a HIFU toolbox based on BEM++ and Dune for this treatment planning simulation. This is work in collaboration with the National Physics Lab and is of large industrial interest for the development of novel cancer therapies. The second medical application is in functional neuroimaging. An example is diffuse optical cortical mapping, where we use the library to simulate the diffuse scattering of light in realistic head geometries. These techniques are useful to map the brain activities of patients and can have potential applications for example in the treatment of sufferers of locked-in syndrome.

The other large application area studied in this proposal is atmospheric sciences in collaboration with the Met Office. The interest here is to accurately simulate the electromagnetic properties of atmospheric particles such as ice crystals. Remote sensing of cloud properties relies heavily on electromagnetic scattering models and questions regarding the accuracy of standard approaches have potentially profound implications for our understanding of how the water cycle and clouds influence the radiation balance of the earth. BEM++ computations of scattering by these atmospheric particles will be compared with results from previously used asymptotic methods, and the implications for remote-sensing and radiative transfer in the atmosphere investigated.

Planned Impact

High-quality open source software forms the fundamental component of modern scientific computing. Modern high-performance finite element libraries such as deal.II or DUNE allow the solution of complex simulation problems on thousands of cores. But while there exist a number of high-performance libraries for finite element discretizations on massively parallel systems, only little is available for the solution of boundary integral equations. Yet, boundary integral equations are important tools in a wide range of applications, such as medical imaging, atmospheric modelling, and electro-magnetic or acoustic shielding in the aeronautical and automotive industries.

BEM++ is closing this gap by delivering a free, highly performant modern boundary element library that can be easily adopted for boundary integral equation problems in all areas of science and engineering. In Stage 1 of the project we have developed the core library foundations. In Stage 2 we present a wide program to i) extend the library for very large problems on heterogeneous computing clusters, ii) interface it with the well established DUNE library to solve complex coupled finite element and boundary element problems (FEM/BEM coupling), and iii) apply the library to complex applications in medical imaging and atmospheric sciences. This work will bring significant impact in several ways.

As a general tool for solving complex coupled boundary integral equation/partial differential equation problems BEM++, in combination with DUNE, will give researchers across science and engineering a highly performant software toolbox to solve large-scale problems in homogeneous or coupled homogeneous and heterogeneous media, enabling them to perform simulations that otherwise would not have been possible for them. Similarly, industry will profit from the availability of a highly performant library for coupled FEM/BEM simulations, which can be used for in-house simulations or integrated into their products.

Significant impact will also be generated from the applications worked on during the course of this proposal. A large application area is for medical imaging problems. Here, we will work on two areas, namely High-intensity Focused Ultrasound (HIFU), and functional neuroimaging. In HIFU focused ultrasound is used to destroy cancerous tissue. In order to generate a treatment plan for patients, careful numerical simulations need to be performed to target the ultrasound correctly. Based on BEM++ and DUNE, and with collaboration with the National Physics Lab, we will build a HIFU software toolbox, which will make possible fast and accurate simulations for treatment planning. This application is of direct industrial relevance and may lead to further commercialization. Another application is functional neuroimaging. In Stage 1 we have already started work on Diffuse Optical Cortex Mapping and will continue this work and include other functional imaging modalities. Functional neuroimaging is one of the most fruitful and challenging areas of applied research. Mapping neural activity has potential in a range of applications, from helping sufferers of "locked-in" syndrome to the development of brain-interface devices. Efficient tools to simulate functional neuroimaging modalities are essential to drive this area forward.

Another application area targeted in Stage 2 is the large-scale simulation of electromagnetic scattering by atmospheric particles in collaboration with the Met Office. This work will help the verification of currently used models for atmospheric particles, and provide a pathway into novel applications of boundary elements for the remote sensing of the atmosphere.

Many other applications of the library are likely arise over time, thanks to its generic and polymorphic nature, and will lead to impact in areas of strategic importance to EPSRC such as "Manufacturing the Future".

Publications

10 25 50
publication icon
Bespalov A (2019) Adaptive BEM with optimal convergence rates for the Helmholtz equation in Computer Methods in Applied Mechanics and Engineering

publication icon
Schweiger M (2016) Basis mapping methods for forward and inverse problems in International Journal for Numerical Methods in Engineering

publication icon
Kleanthous A (2019) Calderón preconditioning of PMCHWT boundary integral equations for scattering by multiple absorbing dielectric particles in Journal of Quantitative Spectroscopy and Radiative Transfer

publication icon
Groth S (2015) The boundary element method for light scattering by ice crystals and its implementation in BEM++ in Journal of Quantitative Spectroscopy and Radiative Transfer

publication icon
Betcke T (2017) Overresolving in the Laplace Domain for Convolution Quadrature Methods in SIAM Journal on Scientific Computing

publication icon
Van 't Wout E (2015) A fast boundary element method for the scattering analysis of high-intensity focused ultrasound. in The Journal of the Acoustical Society of America

 
Description Bempp has become a widely used tool for boundary element computations. We have a good user community and a number of publications associated with Bempp. The original paper describing the library has well over 100 citations already.
Exploitation Route We are soon releasing a complete redevelopment of Bempp, which is CPU and GPU capable. It is based on OpenCL, allowing it to run on AMD and Nvidia GPUs. On CPUs the new version uses AVX and AVX-512 vector units of modern CPUs.

Bempp is used in two further EPSRC Grants and one current Met Office CASE Studentship. We furthermore have a joint collaboration grant with PUC Chile.
Sectors Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Environment,Healthcare

URL http://www.bempp.com
 
Description The Bempp library and its successor Bempp-cl are actively used across natural sciences and engineering, as citations of our relevant papers show. Over the years the library has also led to a number of successful consultancies with industrial partners. As part of EPSRC's Excalibur initiative for Exascale software we are now building a new platform based on our experience with Bempp that will be scalable across massively parallel systems and allow novel applications that can fully utilize the power of upcoming generations of exascale architectures.
Sector Digital/Communication/Information Technologies (including Software),Environment,Healthcare
Impact Types Economic

 
Description NERC Industrial Case Studentships
Amount £90,000 (GBP)
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 10/2016 
End 09/2020
 
Description Optimising patient specific treatment plans for ultrasound ablative therapies in the abdomen (OptimUS)
Amount £706,803 (GBP)
Funding ID EP/P012434/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2017 
End 03/2020
 
Description Reducing the Threat to Public Safety: Improved metallic object characterisation, location and detection
Amount £570,000 (GBP)
Funding ID EP/R002274/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2018 
End 12/2020
 
Description Met-Office Collaboration 
Organisation Meteorological Office UK
Country United Kingdom 
Sector Academic/University 
PI Contribution We have provided expertise on the numerical simulation of electromagnetic waves using our BEM++ software.
Collaborator Contribution They provided application expertise for atmospheric application, benchmark geometries and publication expertise in corresponding subject specific journals.
Impact - The joint paper: Groth, S. P., Baran, A. J., Betcke, T., Havemann, S., & Smigaj, W. (2015). The boundary element method for light scattering by ice crystals and its implementation in BEM plus. JOURNAL OF QUANTITATIVE SPECTROSCOPY & RADIATIVE TRANSFER, 167, 40-52. doi:10.1016/j.jqsrt.2015.08.001 - A successful NERC Case Studentship application on boundary element methods for light scattering problems. The successful grant outcome has been announed in December 2015 and we are currently recruiting a joint PhD student together with the Met Office.
Start Year 2014
 
Title BEM++ 3.1 
Description The new version of our main BEM++ software library. It has now extended Maxwell functionality and supports MPI parallelization. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact We use this software as part of an ongoing industrial collaboration and as basis for further commercialisation 
URL http://www.bempp.org
 
Title BEM++ v3.0 
Description BEM++ v3.0 is a major redevelopment of BEM++ and the main outcome of the BEM++ Stage 2 proposal. The new developments in BEM++ 3.0 are as follows: - A new fully cmake based installer than works on a variety of Linux and Mac systems. Optional a Homebrew installer for Mac and binary packages for Ubuntu are provided. For unsupported Linux distributions or Windows systems a Virtualbox virtual machine image is now provided as default to distribute the software. This significantly improved the user experience and allows to use BEM++ within a few minutes from downloading. - Reduction of external dependencies. We have cleaned up the code base and removed several external dependencies, making the software much easier to install and maintain. - Redesign of large parts of the library in Python. Originally BEM++ was developed mostly in C++ with a thin Python interface on top. We have reimplemented all non-performance critical parts of the library directly in Python, leaving only low-level assembly loops and multithreaded routines in C++. This has significantly improved the maintainability and extendability of BEM++. One of the postdocs on the project has implemented fast high-frequency preconditioners purely from Python without requiring the need to work in C++. - Development of our own H-Matrix library. Previously, BEM++ depended on the external AHMED software for fast assembly of large-problems. AHMED has a difficult, not open-source compliant license. We have solved this problem by developing our own hierarchical matrix solutions, which performs excellently and is now fully integrated in BEM++. We have solved very large high-frequency problems from high-intensity focused ultrasound treatment with our new library and achieve excellent performance. - Development of various preconditioning techniques. We have implemented high-frequency preconditioners for Helmholtz and opposite order preconditioners for Laplace type problems. These are useful in large-scale application problems. The high-frequency preconditioners have already been used in a real world data engineering application. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact While it is too early to gather precise impact data we are pleased that the main paper describing the BEM++ library, which has appeared in Transactions on Mathematical Software in 2015, has been cited 40 times (Google Scholar citations) in its first year, which is a very high number for Mathematics. Also, we regularly observe now other groups at conferences presenting results based on BEM++. As time progresses the citation count of the main BEM++ paper will be some measure on how widely the software continues to be used. 
URL http://www.bempp.org