Constructing a Digital Twin for a self-correcting Scanning Transmission Electron Microscope using Machine Learning Approaches

The goal of this PhD project is to develop a full digital twin model for the Scanning Transmission Electron Microscope (STEM) to enable in silico optimisation of experiments, reconstruction algorithms and the adaptive sub-sampling approach needed to generate unique scientific insights across the length and timescales of the instrument function (microns to atoms and picoseconds to hours). Included in this development will be statistical emulation using a multi-output Gaussian Process (MOGP), which enables machine learning algorithms to consider uncertainty across multiple signals, which in this case will be the many correlated images/spectra generated by a single experiment in the microscope. From this development it is anticipated that image optimisations such as focus, tilt, stigmation and aberrations can be self-compensated for during acquisition, taking the key first steps to self-driving experimental approaches to materials characterisation.

The initial applications of this technology have focused on electron microscopy, where a real time acquisition mode for atomic resolution images/spectroscopy has been developed in which inpainting reconstructions are aided by a critical deep learning step - the microscopes are learning how to take the best images for themselves and then optimising the experimental acquisition. SenseAI is now working with several major instrument manufacturers to broaden these new approaches to instruments using X-rays, ions, neutrons and optics in addition to the existing portfolio of electron microscopes, with the goal of developing self-driving acquisition and analysis capabilities in the near future.