A High-Order Model of the Earth's External and Induced Magnetic Field

Lead Research Organisation: British Antarctic Survey
Department Name: Science Programmes

Abstract

For centuries people have used magnetic compasses to guide them on their way and explore new territories. This has led scientists to embark on their own journeys of discovery about Earth's magnetism, and to the discovery of electromagnetism that is at the heart of modern technology - phones, TVs, computers, etc. Now, in the age of GPS, you might think that compasses are obsolete, but guidance by the Earth's magnetic field is still vital to explore for oil and minerals below ground (where GPS can't reach) and as a safety backup for planes, etc. And, ironically, GPS is affected by natural hazards caused by the Earth's magnetic field.
So the scientific study of Earth's magnetism continues to be important in many ways, so much so that in 2012 the European Space Agency will launch a mission called Swarm in which three satellites will orbit the Earth to survey its magnetic field in unprecedented detail. These measurements will be used to improve mathematical models of the geomagnetic field that provide a standard reference for various applications. One target area is a better understanding and description of the relatively rapid and complex magnetic fluctuations caused by electrical currents flowing in the upper atmosphere and in Space, ultimately driven by disturbances happening on the Sun that wax and wane with an 11-year solar cycle. This so-called external magnetic field also induces currents to flow in oceans and under the Earth's surface which in turn creates additional magnetic fluctuations.
Together, the external and induced magnetic field (EIMF) limits the accuracy of geomagnetic field models such that they aren't useful for surveys and navigation at places and times when the EIMF fluctuations are large, such as in the polar regions and during so-called magnetic storms that may happen once a month and last several days. The EIMF also creates a natural hazard for large-scale electrically conducting systems such as power outages in electricity grids, corrosion in oil pipelines, and even phantom railway signals.
In this project we will study the EIMF using a solar cycle's worth of measurements made at over 300 different locations around the world, recently collected together for the first time by an international project called SuperMAG. Our idea is to borrow mathematical techniques usually used by meteorologists for studying the weather and climate to identify the natural cycles and patterns of the EIMF. In conjunction with the Swarm mission, the resulting new descriptions and understanding of the EIMF "weather" and "climate" should help to improve the next generation of computer models of Earth's magnetic field. It can also be used to as a basis to assess and predict the risk of power outages in UK's National Grid caused by extreme EIMF fluctuations.

Planned Impact

The main beneficiaries of our research will be the following:
a. Surveyors of natural resources. Surveys to explore for natural underground resources use models of the geomagnetic field to guide their equipment. These models neglect most of the EIMF, which means that they are unsuitable for surveying during geomagnetically active times and in the polar regions. Thus geomagnetic field modellers are interested in ways to better represent the EIMF in their models for such stakeholders (see letter of interest from Prof Nils Olsen, author of the Comprehensive Model of the geomagnetic field). In addition, information on past EIMF variability and its controlling factors can help the exploration industry manage the risks of disruption to their surveys.
b. Electricity supply network managers. Rapid external magnetic field variations cause Geomagnetically Induced Currents (GIC) to flow in high voltage power grids, in polar, mid and low latitude power systems. Uncertainty in our understanding of GIC gives rise to large risks, with the UK government concluding that the potential impact of severe such space weather on the national infrastructure is "one of the highest priority risk areas" [H.M. Government, 2010]. There is thus an interest from electricity supply network managers in knowledge and models of GICs that they can use to develop strategies to manage the risk.
c. Society. Scientific discovery and knowledge enriches society through its inspiration and education. BAS attracts a particularly high media profile with 3,625 individual news items (in the English language) during 2009, providing over half a billion people with the opportunity to see/hear/read about BAS in the UK and at least 65 other countries. 70% of these were generated pro-actively from 16 press releases mostly (13) based on peer-reviewed science publications. Lancaster University provides a service called AuroraWatch to over 48,000 people, providing them with email messages, tweets and Facebook updates alerting them to the possibility of seeing the aurora in the UK, based on real-time data from SAMNET magnetometers. Lancaster University also provides undergraduate education in space weather and runs educational activities for the local community. All these will contribute to raising public awareness of geomagnetism and its space weather effects.
d. Local industry. BAS has developed a special low power magnetometer (LPM) system suitable for remote environments based on solar and wind power, and innovative instrument and data management. The power supply technology is also now being used to run other instruments such as GPSs, meteorological stations, and remote cameras. Besides the 11 BAS LPMs, BAS has also made 12 other LPMs under contract to China, Italy and Japan (4 each). Fabrication has now been spun-off to a local company who have to date delivered the Chinese contract and several other power systems, generating over £20k income to NERC. As part of this proposal, BAS will develop a real-time communications capability for the LPMs.
 
Description The main outcome of this award has been a compact mathematical description of the surface magnetic field above 50 degrees North over the last solar cycle (1997-2009) in terms of its main dynamical modes, the discovery of how the relative importance of each mode changes from month to month, and how predictable each mode is from early-warning spacecraft measurements in the solar wind upstream of the Earth. The model is available at dx.doi.org/10.5285/4013dcb3-5151-44ae-9bae-885943139600.
In this grant we have used a clever mathematical method called Empirical Orthogonal Function (EOF) analysis to find that the seemingly complicated variations in the magnetic field measured by about 300 magnetometers around the globe is mostly made up of just a few relatively simple spatial patterns that vary in time. The temporal variation of some of these modes is found to be closely related to plasma and magnetic field variations measured by a spacecraft in the solar wind upstream of the Earth.
Discovering these modes allows us to better estimate the magnetic field variations at any location on Earth, away from the measurement locations. In this way, we have produced a complete description of the surface magnetic field above 50 degrees North over the last solar cycle (1997-2009). It also means that we can predict much of the variability of the surface magnetic field from solar wind measurements with about 1 hour advance warning, and also assess where and when this prediction is better or worse.
This should be useful for understanding and predicting hazards to the National Grid and other large electrical infrastructures such as railways and pipelines that are caused by such surface magnetic field variability.
Our findings are reported in the following papers:
http://onlinelibrary.wiley.com/doi/10.1002/2015JA022066/full
http://onlinelibrary.wiley.com/doi/10.1002/2016JA023682/full
http://onlinelibrary.wiley.com/doi/10.1002/2017JA024420/full
Exploitation Route The complete description of the surface magnetic field above 50 degrees North over the last solar cycle (1997-2009) will be a valuable open resource for scientists to use in space weather research (dx.doi.org/10.5285/4013dcb3-5151-44ae-9bae-885943139600).
The EOF method, discovered modes, and their dependencies on the solar wind, will be useful for understanding and predicting hazards to the National Grid and other large electrical infrastructures such as railways and pipelines that are caused by surface magnetic field variability.
Sectors Aerospace, Defence and Marine,Energy,Environment,Transport

URL http://onlinelibrary.wiley.com/doi/10.1002/2016JA023682/full
 
Description The National Space Security Policy has the aim "To make the United Kingdom more resilient to the risk of disruption to space services and capabilities, including from space weather", a natural hazard that appears on the National Risk Register. To deliver this resilience, NERC and STFC launched a £20 million, four-year strategic programme called SWIMMR (Space Weather Instrumentation, Measurement, Modelling and Risk) to improve the UK's capabilities for space weather monitoring and prediction. In the NERC grant SWIMMR SAGE running from June 2020 to March 2023, BAS is collaborating with the British Geological Survey and others to develop a model to predict extreme geomagnetically induced currents in electrically conducting infrastructure in the UK. The resultant model will be evaluated for use by the Met Office as a service to the National Grid, and pipeline and railway operators in the UK. This model is based on forecasts of space weather-driven geomagnetic field variations in the UK developed from the model and knowledge developed in the Current Award (NE/J020796/1 A High-Order Model of the Earth's External and Induced Magnetic Field).
First Year Of Impact 2019
 
Description NERC Standard grant
Amount £321,724 (GBP)
Funding ID NE/N01099X/1 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 06/2016 
End 05/2019
 
Title An empirical orthogonal function reanalysis of the northern polar external and induced magnetic field during solar cycle 23 
Description A netcdf-formatted file containing the original binned data (described in Shore et al [2017], doi pending), in their state before they were subjected to EOF analysis. These have had additional processing applied to the SuperMAG data (publically available at http://supermag.jhuapl.edu/) in the form of sampling them to the centroid of the bins, thus they are worth providing here despite the large file size (approximately 12GB). To conserve file space, we have removed empty bins, thus the temporal and spatial basis for these data are provided for each filled bin element. Please note that the binned data had not had the temporal mean values (described in Shore et al [2017], doi pending, and available in the Supporting Information) removed when they were stored in this netcdf file. The file contains 144 (monthly) sets of 8 variables. These variables are named: 1: filled_bin_data_YYYYMM_r 2: filled_bin_data_YYYYMM_theta 3: filled_bin_data_YYYYMM_phi Variables 1 to 3 contain the nanoTesla vales of the binned data for each of the three magnetic field components in the Quasi-Dipole frame. 4: filled_bin_contrib_stations_YYYYMM The three-letter SuperMAG acronym of the station which contributed to each 5-minute mean data point. 5: filled_bin_colats_YYYYMM 6: filled_bin_longs_YYYYMM Variables 5 and 6 are the co-latitude and longitude coordinates of each filled bin element. 7: filled_bin_times_YYYYMM The 5-minute-mean epoch of each filled bin element, with columns in the order: year, month, day, hour, minute, second). 8: filled_bin_indices_YYYYMM A set of fiducial values describing how the sparse elements of the 1D vector of filled bin values relate to the fiducials of the (transposed!) EOF prediction a 2D matrix product of the spatial and temporal eigenvectors with values in every bin. An example of the usage of these data is given in the MATLAB program Shore-ms01.m, provided in the Supporting Information of Shore et al [2017], (doi pending). 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
 
Title An empirical orthogonal function reanalysis of the northern polar external and induced magnetic field during solar cycle 23 - VERSION 2.0 
Description A netcdf-formatted file containing the original binned data (described in Shore et al [2017], doi pending), in their state before they were subjected to EOF analysis. These have had additional processing applied to the SuperMAG data (publically available at http://supermag.jhuapl.edu/) in the form of sampling them to the centroid of the bins, thus they are worth providing here despite the large file size (approximately 12GB). To conserve file space, we have removed empty bins, thus the temporal and spatial basis for these data are provided for each filled bin element. Please note that the binned data had not had the temporal mean values (described in Shore et al [2017], doi pending, and available in the Supporting Information) removed when they were stored in this netcdf file. The file contains 144 (monthly) sets of 8 variables. These variables are named: 1: filled_bin_data_YYYYMM_r 2: filled_bin_data_YYYYMM_theta 3: filled_bin_data_YYYYMM_phi Variables 1 to 3 contain the nanoTesla vales of the binned data for each of the three magnetic field components in the Quasi-Dipole frame. 4: filled_bin_contrib_stations_YYYYMM The three-letter SuperMAG acronym of the station which contributed to each 5-minute mean data point. 5: filled_bin_colats_YYYYMM 6: filled_bin_longs_YYYYMM Variables 5 and 6 are the co-latitude and longitude coordinates of each filled bin element. 7: filled_bin_times_YYYYMM The 5-minute-mean epoch of each filled bin element, with columns in the order: year, month, day, hour, minute, second). 8: filled_bin_indices_YYYYMM A set of fiducial values describing how the sparse elements of the 1D vector of filled bin values relate to the fiducials of the (transposed!) EOF prediction a 2D matrix product of the spatial and temporal eigenvectors with values in every bin. An example of the usage of these data is given in the MATLAB program Shore-ms01.m, provided in the Supporting Information of Shore et al [2017], (doi pending). This VERSION 2.0 data set has been corrected for a bug which led to the bins which span the local midnight meridian having fewer samples than they should. The data density in these bins is now in-line with the rest of the polar coverage. Apart from that change, the original and updated data sets are the same. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
 
Title Geomagnetic Reanalysis Product -- Model of Surface External and Induced Magnetic Field 
Description This study processes over a decade of ground-based magnetometer data at 5 min resolution to arrive at a new model for the magnetic field external to the Earth's surface. We applied a meteorological analysis method called Empirical Orthogonal Functions to the magnetometer data in order to fulfill two goals: (1) we infill the gaps in the available data with solutions that depend on the data alone, rather than on modeling assumptions, thus improving the infill accuracy with regards to available alternative methods. (2) we decompose the infilled data into independent spatial and temporal patterns, each of which describe the maximum possible data variance of any possible pattern. We need these because the structure of the patterns-which is unknown prior to doing the analysis-provides insight into the geomagnetic perturbations at ground level. This whole process resulted in 1,440 separate magnetic field models. Lastly, we applied the method of network analysis to these models in order to systematically classify their spatiotemporal patterns, and provide a clear overview of geomagnetic variations spanning an 11 year solar cycle. In this way, we resolved the geometry of the ionospheric equivalent electrical current systems causing the signals recorded by the magnetometers, and thus gained the best-yet description of how these equivalent currents vary in space and time. The model is described in full in the paper here http://onlinelibrary.wiley.com/doi/10.1002/2017JA024420/full, where it is available to download from the supporting information archive. The datasets from which the model was produced are available at https://data.bas.ac.uk/full-record.php?id=GB/NERC/BAS/PDC/00935, with DOI 10.5285/4013dcb3-5151-44ae-9bae-885943139600. 
Type Of Material Data analysis technique 
Year Produced 2018 
Provided To Others? Yes  
Impact Two ongoing MSc projects, involving international collaboration with satellite atmospheric drag researchers. 
URL http://onlinelibrary.wiley.com/doi/10.1002/2017JA024420/full
 
Description DTU NERC grant project partner 
Organisation Technical University of Denmark
Department National Space Institute
Country Denmark 
Sector Academic/University 
PI Contribution Financial support for Prof Nils Olsen to visit UK.
Collaborator Contribution Expertise on magnetic field coordinate systems and provision of associated computer code.
Impact Shore R, Whaler K, Macmillan S, Beggan C, Velímský J..Olsen N. (2016). Decadal period external magnetic field variations determined via eigenanalysis. Journal of Geophysical Research: Space Physics, 121 (6), pp. 5172-5184
Start Year 2012
 
Description International Space Science Institute project 
Organisation International Space Science Institute (ISSI)
Country Switzerland 
Sector Academic/University 
PI Contribution Contributed to scientific discussions at the workshop and the writing of two chapters in Space Science Reviews journal - see below.
Collaborator Contribution Hosted a workshop.
Impact Two chapters in Space Science Reviews journal: Thébault, E., Lesur, V., Kauristie, K. et al. Space Sci Rev (2016). doi:10.1007/s11214-016-0309-5. Finlay, C.C., Lesur, V., Thébault, E. et al. Space Sci Rev (2016). doi:10.1007/s11214-016-0285-9.
Start Year 2015
 
Description Lancaster University NERC grant partner 
Organisation Lancaster University
Department Lancaster Environment Centre
Country United Kingdom 
Sector Academic/University 
PI Contribution Advice on their scientific research
Collaborator Contribution Advice on our scientific research
Impact Shore, R. M., M. P. Freeman, J. A. Wild, and J. W. Gjerloev (2017), A high-resolution model of the external and induced magnetic field at the Earth's surface in the Northern Hemisphere, J. Geophys. Res. Space Physics, 122, doi:10.1002/2016JA023682.
Start Year 2012
 
Description SuperMAG data consultancy 
Organisation University of Bergen
Department Department of Biology
Country Norway 
Sector Academic/University 
PI Contribution Use of SuperMAG data for scientific research
Collaborator Contribution Expert advice on the use of ground magnetometer data from the SuperMAG data archive http://supermag.jhuapl.edu/
Impact Scientific paper: Shore R, Freeman M, Wild J, Gjerloev J. (2017). A high-resolution model of the external and induced magnetic field at the Earth's surface in the Northern Hemisphere. Journal of Geophysical Research: Space Physics, Not multi-disciplinary.
Start Year 2012
 
Description Cambridge Science Festival 2015 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact 'Look in to the Polar Light' stand at the Scott Polar Research Institute as part of Cambridge Science Festival on Saturday 14th March 2015. Museum staff reported good feedback from attendees. Feedback from public suggested that they attended because this was the only aurora-themed event for that year's festival.
Year(s) Of Engagement Activity 2015
 
Description New Scientist Live 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Major annual national event organised by New Scientist magazine held at London ExCel centre, September 2016. We hosted a space weather stand there as part of a BAS team. We provided a Magic Planet installation to visualise our science to the audience (http://globalimagination.com/).
Year(s) Of Engagement Activity 2016
URL https://live.newscientist.com/2016-show/exhibitors/