CCP4 Advanced integrated approaches to macromolecular structure determination
Lead Research Organisation:
STFC - LABORATORIES
Department Name: Scientific Computing Department
Abstract
Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
Planned Impact
The generic importance of macromolecular crystallography in general and CCP4 in particular is provided in the Pathways to Impacts section.
Electron diffraction (ED) on 3D crystals represents a promising new area for macromolecular structural biology. The strength of the interaction means that the technique is suited to very small samples (nanocrystals), from which many rotation images can be collected before radiation damage limits diffraction. This makes the method extremely competitive with XFEL serial crystallography, with costs being orders of magnitude lower. Structural information from ED is complementary to that from X-ray diffraction, with ED able to locate hydrogen positions and assess the charged state of residues and ions in the sample. One of main factors preventing routine high quality structure determination by ED is difficulty in the interpretation of experimental data due to dynamic diffraction effects (multiple scattering of electrons). Currently, methods to model dynamic diffraction require a complete structural model and a high level of expertise to use. As a result, ED for macromolecules currently remains limited to use in extra-thin samples, where the dynamic diffraction component may be neglected.
Successful accomplishment of WP4 will have the following impacts:
1) An efficient tool for the mathematical modelling of ED experiments will be created. This will further the understanding of effects relevant to ED experiments, leading to optimization of experimental protocols and choice of sample crystals
2) The manifestation of dynamic diffraction will be studied in detail sufficient to understand the limits of the ED technique in terms of crystal size, symmetry, plus the effect of mosaicity and disorder, which has a mitigating effect on multiple scattering.
3) This understanding will inform procedures for extrapolating ED intensities obtained from multiple crystals to the kinematic approximation limit. In this regime, existing MX data processing software may be used for scaling and merging of ED images. This would enable more regular handling of ED data through phasing and model building with established software packages for MX, such as CCP4.
4) An immediate practical impact will be in the development of "ED scaling" algorithms within the dials.scale module of the DIALS data-processing software, developed jointly by Diamond Ltd., CCP4 and the Lawrence Berkeley National Laboratory in California. This will allow researchers to process ED data using familiar tools from X-ray diffraction approach.
Electron diffraction (ED) on 3D crystals represents a promising new area for macromolecular structural biology. The strength of the interaction means that the technique is suited to very small samples (nanocrystals), from which many rotation images can be collected before radiation damage limits diffraction. This makes the method extremely competitive with XFEL serial crystallography, with costs being orders of magnitude lower. Structural information from ED is complementary to that from X-ray diffraction, with ED able to locate hydrogen positions and assess the charged state of residues and ions in the sample. One of main factors preventing routine high quality structure determination by ED is difficulty in the interpretation of experimental data due to dynamic diffraction effects (multiple scattering of electrons). Currently, methods to model dynamic diffraction require a complete structural model and a high level of expertise to use. As a result, ED for macromolecules currently remains limited to use in extra-thin samples, where the dynamic diffraction component may be neglected.
Successful accomplishment of WP4 will have the following impacts:
1) An efficient tool for the mathematical modelling of ED experiments will be created. This will further the understanding of effects relevant to ED experiments, leading to optimization of experimental protocols and choice of sample crystals
2) The manifestation of dynamic diffraction will be studied in detail sufficient to understand the limits of the ED technique in terms of crystal size, symmetry, plus the effect of mosaicity and disorder, which has a mitigating effect on multiple scattering.
3) This understanding will inform procedures for extrapolating ED intensities obtained from multiple crystals to the kinematic approximation limit. In this regime, existing MX data processing software may be used for scaling and merging of ED images. This would enable more regular handling of ED data through phasing and model building with established software packages for MX, such as CCP4.
4) An immediate practical impact will be in the development of "ED scaling" algorithms within the dials.scale module of the DIALS data-processing software, developed jointly by Diamond Ltd., CCP4 and the Lawrence Berkeley National Laboratory in California. This will allow researchers to process ED data using familiar tools from X-ray diffraction approach.
Publications
Agirre J
(2023)
The CCP4 suite: integrative software for macromolecular crystallography.
in Acta crystallographica. Section D, Structural biology
Aragon M
(2024)
Applying 3D ED/MicroED workflows toward the next frontiers.
in Acta crystallographica. Section C, Structural chemistry
Drevon T
(2021)
Simulation of electron diffraction patterns of organic crystals under continuous rotation
in Acta Crystallographica Section A Foundations and Advances
Drevon TR
(2023)
Dynamical diffraction of high-energy electrons by light-atom structures: a multiple forward scattering interpretation.
in Acta crystallographica. Section A, Foundations and advances
González-González J
(2024)
Molecular structure and selective theophylline complexation by conformational change of diethyl N , N '-(1,3-phenylene)dicarbamate
in Acta Crystallographica Section C Structural Chemistry
Klar P
(2024)
Cryo-tomography and 3D Electron Diffraction Reveal the Polar Habit and Chiral Structure of the Malaria Pigment Crystal Hemozoin
in ACS Central Science
Kovalevskiy O
(2021)
Helping researchers to solve their structures: automation and user guidance in CCP4 Cloud
in Acta Crystallographica Section A Foundations and Advances
Krissinel E
(2022)
CCP4 Cloud for structure determination and project management in macromolecular crystallography.
in Acta crystallographica. Section D, Structural biology
Waterman DG
(2023)
A standard data format for 3DED/MicroED.
in Structure (London, England : 1993)
| Title | Bloch tool showcasing video |
| Description | Video describing the use of Bloch simulation tools and the physics behind the kinematic approximation in electron scattering and the theory of dynamical scattering based on Bloch wave approach |
| Type Of Art | Artefact (including digital) |
| Year Produced | 2021 |
| Impact | The video is created for educational purposes |
| URL | https://www.ccp4.ac.uk/ccp4-ed/documents/bloch_tools.mp4 |
| Title | Quantum mechanics of crystal video series |
| Description | Detailed animated explanations of Blochwave theory and quantum mechanics of crystal matter, aimed at a undergraduate to postgraduate audience |
| Type Of Art | Film/Video/Animation |
| Year Produced | 2022 |
| Impact | The series has educational value for general audience wishing to understand the principles of quantum-mechanical approaches to the description of crystal matter and Bloch wave theory of scattering |
| URL | https://www.youtube.com/@crystalpanicii6653 |
| Description | Our aim is to develop a model for electron diffraction from macromolecular samples and perform a series of modelling experiments, the results from which could inform data processing software (specifically, DIALS) of better protocols and parameters for processing electron diffraction images. Ultimately, this is expected to enable further progress in structure solution software development, as well as experimental designs. In particular, we expect to use the developed models for developing and implementing new scaling algorithms for electron diffraction data within DIALS. During the first year, we performed theoretical and computational analysis, comparing two principal approaches to ED modelling. The nearBragg approach is based on the adaptation of the optical diffraction model to systems with significant scattering cross-sections, resulting in multiple scattering.Another approach employs the quantum-mechanical(QM) formalism for the calculation of the scattered electron wave function. Having conducted ED simulations in both frameworks, we found that the nearBragg approach produces noticeable artefacts due to the inadequacy of the classical picture used when representing multiple scattering events. A deeper investigation during the second year based on applying a rigorous real space multiple scattering implementation enabled an accurate determination of the scattering amplitudes necessary to match the results from quantum mechanical calculations. This in-depth investigation revealed that for moderately thick crystals, accurate calculations require high order multiple scattering terms which rapidly make the use of this approach prohibitive even for light-atom molecules such as proteins. This finding is the subject of a manuscript that should be submitted very soon. QM-based approach solving Shroedinger's equation for a many-atom system remains the method of choice for all practical purposes. Following findings in the first year that despite its computational efficiency, the approach known as multislice works only for a subset of preferred crystal orientations which makes it challenging for comparison with actual electron diffraction experiments under continuous rotation. As a result, effort has been dedicated in the second year to producing simulated ED images with yet another approach known as Blochwave which does not suffer this limitation. This has been undertaken for small molecule structures, which are studied experimentally by research groups at UCLA, Stockholm University and the eBIC facility at Diamond Ltd., with their experimental diffraction images made available at the Zenodo resource (https://zenodo.org/). Comparison of simulated and experimental diffraction images for 3 biomolecules: Crambin, Biotin and Ireloh have been conducted, which showed that experimental observations are matched with not a consistent quality: while Biotin results were reproduced nearly perfectly, Ireloh showed noticeable deviations with Crambin in the middle. Preliminary analysis indicates that these results are due to the neglect of unknown experimental key parameters such as crystal thickness, anisotropic Debye Waller factor, inelastic scattering and crystal defects. The effect of crystal thickness is covered by our study rather thoroughly, and our results suggest that the other factors may need to be modelled in some cases. It is also possible, at least theoretically, to determine these parameters via an optimisation procedure in the course of a dynamical refinement. Calibrating our approach to experimental data and providing facility for taking crystal defects, B-factor anisotropy, and inelastic scattering remain an important result and activity necessary for reaching our objectives. In the last year additional efforts have been made on improving the geometrical representation of experimental data and including anisotropic displacement parameters in the dynamical simulations. Improvements over previous simulations have been observed. In addition, a systematic study on the effect of dynamical diffraction as a function of specimen thickness has been undertaken and demonstrated that for unit cells with 10A characteristic lattice constants, dynamical effects become significant for sample thickness over 500A especially for strong reflections in zone axis orientations or in systematic rows. Dynamical simulated data have been used against standard kinematic refinement packages such as shelxl with R factors reaching the typically observed values of 20% for thickness over 500A. Further studies on structures with larger unit cell parameters are under way. It is expected that the kinematic regime should hold for larger thickness. Finally, avenues for modeling inelastic scattering have been considered, some of them offering practical options in production codes. This is of particular interest as it can be used to investigate the claim that the effect of inelastic scattering is to reduce the apparent thickness of crystals which is a commonly accepted hypothesis in the community. |
| Exploitation Route | The outcome informs researchers working in electron diffraction modelling fields of advantages and disadvantages of classical (near-Bragg) and QM based approaches. The developed T-matrix approach extends application of this formalism, known from solid state physics, to the specific field of fast electron scattering on ordered atoms, and can be exploited by other researchers using the corresponding software tools released. The outcome also provide developers of refinement software with a tool for taking multiple scattering effects into consideration, which is necessary for achieving best results at 3D structure reconstruction from ED images, particularly when kinematic approximation fails due to crystal thickness and geometry. |
| Sectors | Education Pharmaceuticals and Medical Biotechnology Other |
| URL | http://www.ccp4.ac.uk/ccp4-ed/ |
| Title | Electron Diffraction Modelling using the T-matrix Formalism |
| Description | T-matrix formalism, borrowed from Solid State Physics, was adapted to the specific problem of fast electrons scattering on ordered atoms. The corresponding theoretical derivations were implemented in a software package released through GitHub, and prepared for publication. |
| Type Of Material | Computer model/algorithm |
| Year Produced | 2021 |
| Provided To Others? | Yes |
| Impact | No impact is available yet |
| URL | https://www.ccp4.ac.uk/ccp4-ed/documents/articles/pyscat.pdf |
| Description | DIALS Software |
| Organisation | Diamond Light Source |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | DIALS Software for diffraction image processing was complemented with module for exporting data in PETS-2 format, which contains complete information about diffraction geometry at each frame of data collection. This data is required for processing ED images with high fraction of dynamical scattering component, studied in the Project, and it was not available before |
| Collaborator Contribution | The developed PETS format module was included in DIALS software and distribution systems. |
| Impact | Complete experimental data is now available for comparison with ED simulation results, which allows for more in-depth analysis and more conclusive results. |
| Start Year | 2022 |
| Description | Lukas Palatinus laboratory Jana software |
| Organisation | Academy of Sciences of the Czech Republic |
| Department | Institute of Physics |
| Country | Czech Republic |
| Sector | Learned Society |
| PI Contribution | We have extended DIALS to output data in the format required by Jana software for dynamic diffraction refinement. This activity is a direct result of our work on computational models for electron diffraction. |
| Collaborator Contribution | We get access to Jana software and their efficient implementation of Blochwave approach for modelling of electron diffraction patterns. This was important for cross-checking results of ED modelling |
| Impact | DIALS data processing software was extended to capture and output data necessary for structure refinement using dynamical scattering effects. This helps to avoid negative effects of kinematic scattering approximation and obtain better refined structures. |
| Start Year | 2021 |
| Description | Modelling the impact of crystal defects on electron diffraction |
| Organisation | University of Bremen |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | We conduct Electron Diffraction simulations of Silicon crystal as the starting point for comparing rocking curves simulated with Blochwave and Multislice simulation methods. This is a preliminary step before including defects in Multislice simulations. |
| Collaborator Contribution | Software package and expertise in the description and modelling of crystal defects |
| Impact | This is a multi-disciplinary collaboration, which includes theory of solid state, quantum theory of electron scattering in crystalline media and advanced numerical methods for simulating effects of crystal defects on electron diffraction. |
| Start Year | 2024 |
| Description | Richard Beanland group, University of Warwick |
| Organisation | University of Warwick |
| Department | Department of Physics |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | The partner contributed their FELIX Bloch wave simulation software into Project, which was then incorporated into the Electron Diffraction simulation suite by the Researcher |
| Collaborator Contribution | The Researcher communicated to the Partner Bloch wave simulator developed in course of the Project, which was adopted by the partner |
| Impact | The collaboration resulted in the improvement of approaches to Bloch wave simlulator for Electron Diffraction, which was in development by both parties. Ideas and codes were shared to mutual benefit of both parties |
| Start Year | 2022 |
| Description | Web-based utilities for the analysis of electron diffraction data |
| Organisation | Stockholm University |
| Country | Sweden |
| Sector | Academic/University |
| PI Contribution | Development of web-based utilities for the analysis of electron diffraction data: - Image viewers for most of the commonly used image formats in ED - Interface for downloading dataset directly from central Zenodo database: https://zenodo.org/communities/microed/ - Simulation and visualisation of rocking curves using blochwave - import experimental dataset processed with PETS, DIALS or XDS. |
| Collaborator Contribution | Experimental data, experiment-side expertise, use cases |
| Impact | Dissemination of research results and providing researchers, working in electron diffraction area, with online tools for computer studying of their systems. Ultimately, this helps in choosing better experimental strategy and analysis of experimental results as well as general guidance of research. |
| Start Year | 2023 |
| Title | Blochwave based python simulation solver package |
| Description | Blochwave based python simulation solver package, also containing a wrapper for FELIX, an efficient fortran implement blochwave based dynamical refinement solver. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2021 |
| Open Source License? | Yes |
| Impact | No impact resulted to date |
| URL | https://pypi.org/project/debloch/ |
| Title | Online Electron Diffraction Simulator |
| Description | A web-tool for making simulations of Electron Diffraction on crystals online. Main features include: o Thickness dependent intensity simulation at arbitrary orientations o Analysis of excitation errors o Analysis of individual reflections o Simulation of rocking curves o Comparison with experimental dataset o Simulation available with Felix solver o Jobs submission to STFC cloud clusters o LACBED patterns for dynamical refinement o Online comparison with Multislice |
| Type Of Technology | Webtool/Application |
| Year Produced | 2022 |
| Open Source License? | Yes |
| Impact | Used by researchers working in Electron Diffraction for preliminary modelling of ED experiments. The tool helps analysing results, planning experiments and optimise use of instrument time. |
| URL | https://www.ccp4.ac.uk/edly/viewer |
| Title | T-matrix Simulation Package for Electron Diffraction |
| Description | Package for solving the scalar wave equation with a linear array of scattering spheres. It offers the possibility to solve for constant potential well and hard spheres i.e. infinite potential. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2022 |
| Open Source License? | Yes |
| Impact | Used by scientist working in Electron Diffraction for preliminary modelling Electron Diffraction experiments, which helps planning and optimal utilisation of instruments time. |
| URL | https://pypi.org/project/pyScatSpheres/ |
| Title | pyScatSpheres: Electron Diffraction modelling with T-matrix formalism |
| Description | Package for solving the scalar wave equation with a linear array of scattering spheres. Possibility to solve for constant potential well and hard spheres i.e. infinite potential. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2021 |
| Open Source License? | Yes |
| Impact | The software features a new method for simulating electron diffraction patterns that is not limited to preferable crystal orientations. This makes it possible to simulate real electron diffraction experiment and train data processing software on data with strong dynamic scattering effects, |
| URL | https://pyscatspheres.readthedocs.io/en/latest/ |
| Description | Diffraction methods seminar at RAL |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | Established a seminar series on diffraction methods for researchers working in this area at RAL. The seminar discusses, on bi-weekly basis, methods related to x-ray, electron and neutron diffraction, the corresponding data processing and modelling issues, as well as the underlying physical principles. |
| Year(s) Of Engagement Activity | 2020 |
| Description | Lunch time bytes at CCP4 Study Weekend |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation and demo was given for the online Blochwave dynamical simulator for electron diffraction, developed in the course of grant-funded work. Follow-up questions from estimated 20 people audience were answered. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://studyweekend.ccp4.ac.uk/ |
| Description | NANED Workshop in Instituto Italiano di Technologia, Pontedera, Italy |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Lecture delivered "Software development for micro-ED" with the demonstration of software tools developed in our research, as well as hand-on sessions on using them. Approximately 15 postgraduate students from Europe participated. The workshop aimed at increasing awareness of theoretical research conducted in the field of Electron Diffraction, specifically for macromolecular targets, and presenting software tools helping in optimising experiments and analysis of results, which aims have been achieved. |
| Year(s) Of Engagement Activity | 2024 |