Solar wind data assimilation - maximising the accuracy of space-weather forecasting
Lead Research Organisation:
University of Reading
Department Name: Meteorology
Abstract
"Space weather" describes changes in the Sun's magnetic field which occur over seconds to days. It can damage space- and ground-based technologies, particularly power, communication and Earth-observation systems. In order to forecast space weather with more than about 1 hour of warning time, it is necessary to accurately forecast the solar wind, the continual flow of material away from the Sun which fills the solar system. At present, telescopic observations of the Sun's surface are used to provide the starting conditions for computer simulations of the solar wind. These simulations propagate conditions all the way from the Sun to Earth, where the space-weather impact can be estimated. There are ongoing efforts to improve solar wind simulations and to make more accurate measurements of the solar wind near the Sun. But spacecraft also routinely make direct measurements of the solar wind far from the Sun, which provide useful additional information that is not presently used to improve forecasts. Experience from terrestrial weather prediction shows that the biggest advance in forecasting ability can be achieved by using the available observations to regularly "nudge" the computer simulations back towards reality.
This observational "nudging" of computer models is called "data assimilation" (DA), and it is at the heart of modern weather forecasting. Accurate weather forecast lead times have advanced about a day a decade, mainly due to advances in DA. Given this success, it is time to fully explore DA capabilities for space weather, in particular the solar wind. Our group has recently made preliminary studies in this area. The proposed work will build on this to develop and test the first ever solar wind data assimilation (SWDA) system using a physics-based, operational forecast simulation of the solar wind. This represents the first effort to apply DA to the solar wind in a manner comparable to terrestrial numerical weather prediction. The solar wind, however, differs from the atmosphere and other geophysical systems in a number of fundamental ways, thus adapting existing DA techniques will involve overcoming a number of scientific challenges. This will form the core science of the proposed work.
In addition to improving space-weather forecasting, the SWDA system will enable cutting-edge space-weather research. One by-product of testing the SWDA system is that we will combine models and observations to produce the most accurate estimate to date of the solar wind conditions back near the Sun, where we are unable to directly make measurements. This will help us to understand which magnetic structures on the Sun are related to different solar wind conditions, serving as a direct observational test for theoretical models of solar wind formation.
The SWDA will also be used to determine where, ideally, we would position spacecraft in the solar wind in order to make the biggest improvements to space-weather forecasting. This will inform the design of future space-weather mission design.
This observational "nudging" of computer models is called "data assimilation" (DA), and it is at the heart of modern weather forecasting. Accurate weather forecast lead times have advanced about a day a decade, mainly due to advances in DA. Given this success, it is time to fully explore DA capabilities for space weather, in particular the solar wind. Our group has recently made preliminary studies in this area. The proposed work will build on this to develop and test the first ever solar wind data assimilation (SWDA) system using a physics-based, operational forecast simulation of the solar wind. This represents the first effort to apply DA to the solar wind in a manner comparable to terrestrial numerical weather prediction. The solar wind, however, differs from the atmosphere and other geophysical systems in a number of fundamental ways, thus adapting existing DA techniques will involve overcoming a number of scientific challenges. This will form the core science of the proposed work.
In addition to improving space-weather forecasting, the SWDA system will enable cutting-edge space-weather research. One by-product of testing the SWDA system is that we will combine models and observations to produce the most accurate estimate to date of the solar wind conditions back near the Sun, where we are unable to directly make measurements. This will help us to understand which magnetic structures on the Sun are related to different solar wind conditions, serving as a direct observational test for theoretical models of solar wind formation.
The SWDA will also be used to determine where, ideally, we would position spacecraft in the solar wind in order to make the biggest improvements to space-weather forecasting. This will inform the design of future space-weather mission design.
Planned Impact
The proposed research will improve long-range space weather forecasting. This will most immediately benefit international space-weather forecast agencies, particularly the UK Met Office and the US Space Weather Prediction Center (SWPC). Improved forecasts will ultimately benefit end users by reducing the risk to vital infrastructure and hardware. These include spacecraft which provide data and services that pervade so many aspects of modern life, in particular, global positioning systems (GPS), communications and Earth observation satellites, which underpin terrestrial weather forecasting. Improved space-weather forecasting is also vital for power network security.
Specifically, our research will benefit;
UK Met Office - The Met Office has now taken on the role of providing operational space weather forecasts. Since the UK Met Office works in partnership with the Space Weather Prediction Center (SWPC) in the USA, this will provide a mechanism for sharing our results with the wider operational space weather community. We anticipate our developed solar wind data assimilation system being integrated into their operational model within five years of publication.
Space-weather forecast end users - satellite and power operators can function more efficiently with accurate space weather forecasts. E.g., National Grid expressed a keen interest in space-weather forecasting at recent community meetings. It is particularly important in terms of scheduling maintenance of power lines.
UK and international space agencies - The UK Space Agency is working with national (RAL Space, Airbus) and international (such as the European Space Agency and the National Aeronautical and Space Administration in the USA) partners to develop the next generation of spacecraft in support of operational space weather forecasting. Our work will provide input to these discussions by quantitatively determining the most effective location and type of instrumentation to deploy on a spacecraft for optimal improvement of solar wind models and their ability to accurately forecast Earth-directed space weather events.
The media and publics - Impact will occur throughout the project and, in particular, immediately on publication of any results. Prior experience tells us that there is a great deal of interest in scientific stories concerning space weather. The University of Reading Press Office was highly effective in generating media (and subsequently public) interest and scheduling interviews with TV, radio, print and online media for previous work. and we will ensure that we work with them to maximise the potential for distribution of our research outputs through these channels.
Policy makers - Space weather is now recognised on the UK government's National Risk Register as a potential severe risk, and the UK government are engaged with the scientific community and general public in determining ways of minimising the associated risks and planning for the consequences of any future potential event. The accuracy with which a forecast can be made determines the latency of any alert and this in turn affects what can be achieved in the time. By improving the modelling of the solar wind through data assimilation, our work will have a direct contribution to updating such policies.
Specifically, our research will benefit;
UK Met Office - The Met Office has now taken on the role of providing operational space weather forecasts. Since the UK Met Office works in partnership with the Space Weather Prediction Center (SWPC) in the USA, this will provide a mechanism for sharing our results with the wider operational space weather community. We anticipate our developed solar wind data assimilation system being integrated into their operational model within five years of publication.
Space-weather forecast end users - satellite and power operators can function more efficiently with accurate space weather forecasts. E.g., National Grid expressed a keen interest in space-weather forecasting at recent community meetings. It is particularly important in terms of scheduling maintenance of power lines.
UK and international space agencies - The UK Space Agency is working with national (RAL Space, Airbus) and international (such as the European Space Agency and the National Aeronautical and Space Administration in the USA) partners to develop the next generation of spacecraft in support of operational space weather forecasting. Our work will provide input to these discussions by quantitatively determining the most effective location and type of instrumentation to deploy on a spacecraft for optimal improvement of solar wind models and their ability to accurately forecast Earth-directed space weather events.
The media and publics - Impact will occur throughout the project and, in particular, immediately on publication of any results. Prior experience tells us that there is a great deal of interest in scientific stories concerning space weather. The University of Reading Press Office was highly effective in generating media (and subsequently public) interest and scheduling interviews with TV, radio, print and online media for previous work. and we will ensure that we work with them to maximise the potential for distribution of our research outputs through these channels.
Policy makers - Space weather is now recognised on the UK government's National Risk Register as a potential severe risk, and the UK government are engaged with the scientific community and general public in determining ways of minimising the associated risks and planning for the consequences of any future potential event. The accuracy with which a forecast can be made determines the latency of any alert and this in turn affects what can be achieved in the time. By improving the modelling of the solar wind through data assimilation, our work will have a direct contribution to updating such policies.
Publications
Barnard L
(2022)
HUXt-An open source, computationally efficient reduced-physics solar wind model, written in Python
in Frontiers in Physics
Barnard L
(2020)
Ensemble CME Modeling Constrained by Heliospheric Imager Observations
in AGU Advances
Barnard L
(2023)
SIR-HUXt-A Particle Filter Data Assimilation Scheme for CME Time-Elongation Profiles
in Space Weather
Barnard L
(2022)
Quantifying the Uncertainty in CME Kinematics Derived From Geometric Modeling of Heliospheric Imager Data
in Space Weather
Chi Y
(2020)
Using the "Ghost Front" to Predict the Arrival Time and Speed of CMEs at Venus and Earth
in The Astrophysical Journal
Chi Y
(2021)
Modeling the Observed Distortion of Multiple (Ghost) CME Fronts in STEREO Heliospheric Imagers
in The Astrophysical Journal Letters
James L
(2023)
Sensitivity of Model Estimates of CME Propagation and Arrival Time to Inner Boundary Conditions
in Space Weather
Description | In order to forecast space weather, it is necessary to accurately model the solar wind, the continually expanding solar atmosphere which fills the solar system. At present, telescopic observations of the Sun's surface are used to provide the starting conditions for computer simulations of the solar wind, which then propagate conditions all the way from the Sun to Earth. But spacecraft also make direct measurements of the solar wind, which provide useful additional information that is not presently used. In this study we use a simple solar wind model to develop a method to routinely "assimilate" spacecraft observations into the model and thus improve space weather forecasts. This data assimilation (DA) approach closely follows that of terrestrial weather prediction, where DA has led to increasingly accurate forecasts. We use artificial and real spacecraft observations to test the new solar wind DA method and show that the error in predicting the near-Earth solar wind can be reduced by around a fifth using available observations. |
Exploitation Route | The data assimilation methods could be used for operational space-weather forecasting. |
Sectors | Aerospace Defence and Marine Energy Other |
Description | Solar wind data assimilation methods are being tested for operational forecasting use at the UK Met Office |
First Year Of Impact | 2023 |
Sector | Government, Democracy and Justice |
Description | SSA P3-SWE-IV.2: USE OF L5 DATA IN CME PROPAGATION MODELS |
Amount | € 450,000 (EUR) |
Funding ID | UKRI/RS02344/TP |
Organisation | European Space Agency |
Sector | Public |
Country | France |
Start | 11/2020 |
End | 02/2023 |
Description | SWEEP - STFC/SWIMMR S4 |
Amount | £450,000 (GBP) |
Funding ID | ST/V00235X/1 |
Organisation | Aberystwyth University |
Sector | Academic/University |
Country | United Kingdom |
Start | 09/2020 |
End | 06/2023 |
Title | BRaVDA solar wind data assimilation scheme |
Description | In order to forecast space weather, it is necessary to accurately model the solar wind, the continually expanding solar atmosphere which fills the solar system. At present, telescopic observations of the Sun's surface are used to provide the starting conditions for computer simulations of the solar wind, which then propagate conditions all the way from the Sun to Earth. But spacecraft also make direct measurements of the solar wind, which provide useful additional information that is not presently used. In this study we use a simple solar wind model to develop a method to routinely "assimilate" spacecraft observations into the model and thus improve space weather forecasts. This data assimilation (DA) approach closely follows that of terrestrial weather prediction, where DA has led to increasingly accurate forecasts. We use artificial and real spacecraft observations to test the new solar wind DA method and show that the error in predicting the near-Earth solar wind can be reduced by around a fifth using available observations. |
Type Of Material | Computer model/algorithm |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Scheme is currently being investigated for operational use by UK Met Office |
URL | https://github.com/University-of-Reading-Space-Science/BRaVDA |
Title | HUXt solar wind model |
Description | Near-Earth solar-wind conditions, including disturbances generated by coronal mass ejections (CMEs), are routinely forecast using three-dimensional, numerical magnetohydrodynamic (MHD) models of the heliosphere. The resulting forecast errors are largely the result of uncertainty in the near-Sun boundary conditions, rather than heliospheric model physics or numerics. Thus ensembles of heliospheric model runs with perturbed initial conditions are used to estimate forecast uncertainty. MHD heliospheric models are relatively cheap in computational terms, requiring tens of minutes to an hour to simulate CME propagation from the Sun to Earth. Thus such ensembles can be run operationally. However, ensemble size is typically limited to 10 to 100 members, which may be inadequate to sample the relevant high-dimensional parameter space. Here, we describe a simplified solar-wind model that can estimate CME arrival time in approximately 0.01 seconds on a modest desktop computer and thus enables significantly larger ensembles. It is a one-dimensional, incompressible, hydrodynamic model, which has previously been used for the steady-state solar wind, but it is here used in time-dependent form. This approach is shown to adequately emulate the MHD solutions to the same boundary conditions for both steady-state solar wind and CME-like disturbances. We suggest it could serve as a "surrogate" model for the full three-dimensional MHD models. For example, ensembles of 10k to 1M members can be used to identify regions of parameter space for more detailed investigation by the MHD models. Similarly, the simplicity of the model means it can be rewritten as an adjoint model, enabling variational data assimilation with MHD models without the need to alter their code. The model code is available as an Open Source download in the Python language. |
Type Of Material | Computer model/algorithm |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | HUXt is used by researchers from multiple institutions in the UK, US, China, Austria, Switzerland, and India have published papers based upon its use. As part of a STFC SWIMMR project, HUXt is currently being transitioned into operational forecasting use at the Met Office. |
URL | https://github.com/University-of-Reading-Space-Science/HUXt |