Fast Pixel Detectors: a paradigm shift in STEM imaging

Lead Research Organisation: University of Glasgow
Department Name: School of Physics and Astronomy

Abstract

The research proposed here aims to develop entirely new ways of imaging in the electron microscope, and to use these methods to study real-world materials problems. To illustrate the power of the methods we propose to develop, we have selected proof-of-principle materials problems from the areas of carbon nanotechnology, life sciences and electronic device structures.
Over the past couple of decades a particular type of electron microscope, the scanning transmission electron microscope (STEM) has become increasingly popular due to its high-spatial and energy-resolution. Much of our understanding of how macroscopic materials properties relate to atomic structure and bonding, and how we can control properties by manipulating these, is a result of the development of techniques to characterise materials on very short length scales. The STEM is not only capable of imaging atoms and observing the structure and crystallographic details of materials, but also in performing spectroscopy on single atoms, allowing atom-by-atom chemistry to be determined. A further key development is the spherical aberration corrector, which has revolutionised the performance of these instruments by overcoming the earlier limitations of electron lenses.
The principle of STEM is the use of electron lenses to focus a beam of electrons to form a small illuminating spot or probe. The probe can be scanned across a sample using a beam deflector. A thin, electron-transparent sample is used, and transmitted electrons can be detector, and the intensity of those detected plotted as a function of the probe position during a two-dimensional scan to form an image. The mostly commonly used detectors are an annular dark-field (ADF) detector which is a detector in the form of a broad annulus that detects relatively high angles of scatter, and bright-field (BF) detectors that collect the unscattered and low-angle scattered electrons. Both these detectors collect the total scattering over a range of scattering angles, and the total intensity is used to form an image. Such an approach neglects the rich information that is contained in the fluctuation in intensity that occurs as a function of scattering angle.
The overarching aim that forms the basis of the current proposal is to use fast pixelated detectors to record the intensity as a function of scattering angle in the detector plane of a STEM, which is effectively a diffraction pattern. By recording each two-dimensional diffraction pattern as a function of probe position in a two-dimensional scan, a four-dimensional data set can be recorded that is the ultimate STEM imaging experiment. Such a rich dataset contains information about the phase shift that results from transmission, about the composition of the sample, the strain in the sample and the three-dimensional ordering in the sample. We propose to develop the methods to record this 4D data set, using fast pixelated detectors, and by developing an optimised direct-detection system, together with the methods to process such datasets to enable physically useful measurements to be made.
We believe the approach we are taking will create a paradigm shift in STEM imaging, and in time will become the standard approach to record data in the STEM. To illustrate the power of the approach, we have identified key materials science questions that we will address with the methods we develop. The applications are: (i) imaging charge transfer in doped nanomaterials; (ii) imaging of soft and radiation sensitive materials, (iii) imaging of electric and magnetic fields in magnetic nanostructures, (iv) 3D composition and structural ordering effects in ceramics; (v) interdiffusion in ceramic and semiconductor heterointerfaces.
The methods developed will be disseminated widely, particularly through their implementation at the EPSRC National Facility for Aberration-Corrected STEM (SuperSTEM) through which a wide range of users will be able to access the new methods.

Planned Impact

We have identified a number of pathways by which the outputs of the proposed research will impact more widely than the more obvious academic research implications. These pathways are identified in the diagram presented in the attached Impact Plan, and are summarised here. The pathways may be identified as being direct (i.e. having an immediate, direct causal potential benefit) or indirect (where a link occurs indirectly by supporting other areas of science research). An obvious direct impact of the proposed research is the further training and development of two post-doctoral researchers in the field of advanced STEM. There has been substantial recent capital investment in STEM instrumentation in the UK (approximately £10M EPSRC funding over the past 5 years alone). Projects that support the training of a generation of researchers capable of getting results of the highest quality from such instruments is critical to fully exploit the capital investment.

The economic direct impact of this research is the potential for commercialisation of its outputs as follows:

(i) Commercialisation of detector technologies. A crucial part of the proposed research is to identify, then design and manufacture a bespoke STEM detector optimised for STEM phase contrast imaging. It is likely that a manufacturer would be interested in licensing such a design. Indeed JEOL, Gatan and Deben are all providing support for this project (see letters of support).
(ii) Commercialisation of data acquisition and processing methods A substantial part of the project will involve creating software to control the instrument during data acquisition, and then to process the large amount of data produced to produce images that can be interpreted in terms of object parameters. Such software would undoubted by of interest to commercial suppliers, and licensing agreements would be sought.

The outputs of this research can also be identified as impacting indirectly in the areas of environment, econmy and education.

(i) Economy and environment. Electron microscope imaging is used by researchers in industry and academia across a wide range of materials research, and indirectly impacts upon the EPSRC challenge themes of energy, digital economy, healthcare technologies and living with environmental change through the powerful characterisation capabilities it can provide for materials that are used in these areas.

For example, developments in nanomaterials form the foundations for the many technologies that are addressing the challenge themes described above. Such developments are impossible without the high-resolution characterisation enabled by electron microscopy. Economic activity in nanotechnology already stands at many billions of $ annually worldwide and a current EPSRC portfolio of £170M of grant funding in the socio-economic theme of nanotechnology.

(iii) Education. Both Oxford Materials and SuperSTEM already engage significantly in outreach activities, described further at http://outreach.materials.ox.ac.uk/index.php. The details of these activities, and how they will be strengthened by the research proposed here, are described further in the Impact Plan.

Publications

10 25 50
 
Description Several new forms of imaging using the novel detectors have now been demonstrated, including phase contrast imaging of ribosomes, high sensitivity imaging of magnetisation in thin metallic films and its correlation with internal strain, and imaging of 3 dimensional ordering in crystals, and low noise/ low dose scanning precession electron diffraction.
Exploitation Route The imaging techniques we are developing will be of widespread use in both physical and biological sciences, for the imaging of a wide range of solid objects. This is now resulting in sales of detectors manufactured by us together with Quantum Detectors Ltd. across the world (currently including Belgium, USA, Germany, Canada) and specialist conference sessions on exactly this topic. There is now also a worldwide effort led from Jülich in Germany to bring software developments in this general area together into a common, open-source platform. The work on Scanning Precession Electron Diffraction is attracting further commercial interest for our partners, including a recent enquiry and test for one of the Nordic Universities. And these techniques are feeding into commercial projects with companies in energy and aerospace materials.
Sectors Aerospace, Defence and Marine,Energy,Healthcare,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology

URL https://www.gla.ac.uk/schools/physics/research/groups/mcmp/researchareas/pixstem/
 
Description Work begun in this grant has contributed to follow on commercialisation work of the detector technology led by Dr Damien McGrouther in collaboration with Quantum Detectors Ltd.. This is now sold as a retractable detector by Quantum Detectors Ltd., Harwell, UK. Installations have now been made at NIST (USA), University of Victoria (Canada), University of Antwerp (Belgium), Max Planck Institute for Solid State Research (Stuttgart, Germany). Work is now ongoing with Nanomegas SPRL to integrate this into their ASTAR scanning precession electron diffraction units. Software developed in the project has been released as Open Source on Gitlab and has been installed in several locations outside of Glasgow. And ideas generated in this code are being used by the libertem project based in FZ-Jülich in Germany.
Sector Digital/Communication/Information Technologies (including Software),Manufacturing, including Industrial Biotechology
Impact Types Economic

 
Description LiberTEM 
Organisation Humboldt University of Berlin
Country Germany 
Sector Academic/University 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation Julich Research Centre
Country Germany 
Sector Public 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation Lawrence Berkeley National Laboratory
Country United States 
Sector Public 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation Oak Ridge National Laboratory
Country United States 
Sector Public 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation University of Cambridge
Country United Kingdom 
Sector Academic/University 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation University of Queensland
Country Australia 
Sector Academic/University 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Description LiberTEM 
Organisation University of Sydney
Country Australia 
Sector Academic/University 
PI Contribution Software codes, technical discussions
Collaborator Contribution Software codes, technical discussions
Impact Standardising file formats for pixelated STEM data, starting to integrate academic software packages into a larger open-source framework
Start Year 2017
 
Title Atomap 
Description Atomap is a Python library for analysing atomic resolution scanning transmission electron microscopy images. It relies in fitting 2-D Gaussian functions to every atomic column in an image, and automatically find all major symmetry axes. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact Is being used by us and others in data analysis of atomic resolution images 
URL http://atomap.org/index.html
 
Title Fast pixelated detector live imaging 
Description Python library for real time processing of pixelated STEM data, allowing for live imaging (1000+ fps) of features while acquiring them during a STEM experiment. Currently it works with the Medipix3 detector, and includes the standard virtual detectors (ADF, BF), and differential phase contrast imaging which enables imaging of magnetic domains. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact Has enabled live imaging and therefore improved data collection for a number of investigations, which will be published shortly. Has also been installed at the new ePSIC facility at Diamond Light Source. And is available to the LiberTEM consortium 
URL https://fast_pixelated_detectors.gitlab.io/fpd_live_imaging/
 
Title Merlin for EM 
Description The Merlin for EM Hybrid Pixel Detector (HPD) is an advanced detector development in the field of Electron Microscopy, combining direct detection of electrons and rapid readout in a pixelated format ideal for applications such as 4D STEM and TEM dynamic imaging. A retractable detector mounting and housing has been designed and manufactured at the University of Glasgow in collaboration with Quantum Detectors Ltd.. Three of these have now been purchased and installed at NIST (Maryland, USA), University of Antwerp (Belgium) and the Max Planck Institute for Solid State Research (Germany). 
Type Of Technology Detection Devices 
Year Produced 2017 
Impact A new imaging product being sold by a UK company, with critical parts being manufactured at the University of Glasgow. 
URL http://quantumdetectors.com/merlin-for-em/
 
Title Merlin interface 
Description Python library for interfacing with a Medipix3 detector through the Merlin readout system. Allows for (almost) full automation of acquisition with the Medipix3. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact Has improved data acquisition on the project and is available to other Merlin / Medipix users. 
URL https://fast_pixelated_detectors.gitlab.io/merlin_interface/
 
Title fpd_data_processing 
Description Python library for post processing of pixelated STEM datasets, which includes virtual detectors (BF, ADF), differential phase contrast through center of mass, radial integration, and many convenience functions. Most importantly, it includes functionality for out-of-core processing, meaning at the whole dataset does not have to be loaded into memory. This is especially useful for the pixelated STEM datasets, which can easily be 50 GB+. Builds upon HyperSpy. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact Will be key to upcoming publications, and is available to all, including via the LiberTEM consortium. 
URL https://fast_pixelated_detectors.gitlab.io/fpd_data_processing/
 
Description 3D Structure Imaging Using 4D STEM 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk by Dr Ian MacLaren at the STEM with Advanced Detectors, Lancefield, Victoria, Australia, September 2018
Year(s) Of Engagement Activity 2018
 
Description Atomic resolution in 3 dimensions using pixelated STEM 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Talk by Dr Ian MacLaren at ePSIC User Meeting, Diamond Light Source
Year(s) Of Engagement Activity 2018
 
Description Enabling new capabilities in electron microscopy through detectors 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk by Dr Damien McGrouther at Workshop on STEM with Advanced Detectors, Lancefield, Victoria, Australia, September 2018.
Year(s) Of Engagement Activity 2018
 
Description Imaging Structure and Magnetisation in New Ways Using 4D STEM 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk by Dr Ian MacLaren at Microscopy and Microanalysis 2018, Baltimore, MD.
Year(s) Of Engagement Activity 2018
 
Description New possibilities for STEM using fast pixelated detectors 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk by Dr Ian MacLaren at the 5th Midwest Imaging and Microanalysis Workshop at Notre Dame, South Bend, IN, USA, May 2018
Year(s) Of Engagement Activity 2018
 
Description STEM imaging of the third dimension using Laue zone scattering at atomic resolution 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Poster Presentation and flash talk at the International Microscopy Congress, in Sydney, Australia, September 2018.
Year(s) Of Engagement Activity 2018