Synergy Algorithms for EarthCARE

Lead Research Organisation: University of Reading
Department Name: Meteorology

Abstract

There is a consensus amongst numerical climate models that the earth is warming, but they differ substantially in the predicted size and global distribution of both the warming and associated change to precipitation. This disagreement is largely attributable to uncertainties in how to represent clouds and aerosols in models; clouds are important for climate because they precipitate and via their interaction with solar and thermal infrared radiation, while aerosols can interact with clouds to modulate both of these processes. It is therefore of the highest priority to test and improve the representation of clouds, precipitation and aerosols in models using detailed observations. In 2013, the European and Japanese Space Agencies (ESA and JAXA) will address this problem directly with the launch of the Earth Cloud, Aerosol and Radiation Explorer (EarthCARE) satellite, carrying a radar, a lidar and narrow- and broad-band radiometers. EarthCARE is a significant advance on NASA's 'A-Train' of satellites; the radar is Dopplerized, so will be able to measure vertical motions in clouds, while the 'high spectral resolution' lidar can derive the vertical distribution of optical properties much more reliably than ordinary lidar. Moreover, the lower orbit means that the radar will be around 4 times more sensitive than the radar in the A-Train. A very exciting aspect of EarthCARE is the potential for synergy: when the instruments are used together, much more accurate and comprehensive estimates of cloud properties are possible. However, formulating computer codes to take account of all the available information in a mathematically rigorous way is very challenging. The PI and PDRA on this project are experts in applying rigorous 'variational' methods to combinations radar, lidar and radiometers, as demonstrated by their recent development of a method for deriving the properties of ice clouds from the A-Train. This work has already revealed serious deficiencies in the clouds predicted by the models of the Met Office and the European Centre for Medium Range Weather Forecasts (ECMWF). In this project, we will undertake the ambitious task of developing a retrieval method that can derive the properties of clouds, precipitation and aerosols simultaneously, using all the instruments available on EarthCARE (except the broad-band radiometers, which would be used as an independent test of the retrievals). This is essential to obtain the best possible estimate of atmospheric properties, and thereby to provide the necessary information to test models. Combining such instruments so comprehensively has never been attempted before, and therefore will be of great interest to other users of multiply instrumented ground-based and spaceborne platforms. We will release our flexible code under an open-source license, so that it can be adapted to other combinations of instruments. An additional advantage to our approach is that it yields reliable estimates of the uncertainties in the retrievals, making them suitable for data assimilation, the method by which weather forecast models are able to incorporate all the observations of the atmosphere into a forecast. ECMWF are world leaders in the science of data assimilation, and are currently working on the problem of how to assimilate cloud retrievals from satellites such as EarthCARE. We will work closely with them to ensure that our data products contain all the necessary information to be used for assimilation, and hence to improve weather forecasts in the future. This work will put the UK in an excellent position to exploit EarthCARE in answering the key scientific questions at the heart of climate prediction. Moreover, the exciting results from the A-Train have shown that radar and lidar must have a long-term future in space, even beyond EarthCARE. This project will place the UK at the forefront of spaceborne radar and lidar research and hence in an ideal position to lead future missions.

Publications

10 25 50
publication icon
Barker H (2011) A 3D cloud-construction algorithm for the EarthCARE satellite mission in Quarterly Journal of the Royal Meteorological Society

publication icon
Battaglia A (2010) Multiple-scattering in radar systems: A review in Journal of Quantitative Spectroscopy and Radiative Transfer

publication icon
Delanoë J (2011) Evaluation of ice cloud representation in the ECMWF and UK Met Office models using CloudSat and CALIPSO data in Quarterly Journal of the Royal Meteorological Society

publication icon
Delanoë J (2014) Normalized particle size distribution for remote sensing application in Journal of Geophysical Research: Atmospheres

publication icon
Hogan R (2012) Radar Scattering from Ice Aggregates Using the Horizontally Aligned Oblate Spheroid Approximation in Journal of Applied Meteorology and Climatology

publication icon
Hogan R (2014) Fast Reverse-Mode Automatic Differentiation using Expression Templates in C++ in ACM Transactions on Mathematical Software

 
Description This project has developed an algorithm for retrieving the properties of ice clouds, liquid clouds and rain simultaneously, using combined cloud radar, lidar and radiometer from space. It has been tested on data from the A-Train instruments, specifically the CloudSat radar, the Calipso lidar and the MODIS radiometer. Moreover, the European Space Agency are planning to use it (after further modification) to produce one of the standard products for their forthcoming EarthCARE satellite.

Additional developments:

- A programming technique has been invented for automatically differentiating computer code in a way that is easy to use while being much faster than equivalent software libraries. This has greatly facilitated the development of the retrieval algorithm. The software library "Adept" is available under a free-software license, and is applicable to a wide range of optimization problems.

- We have developed a new approach (the "Self Similar Rayleigh Gans Approximation") to the challenging problem of computing the radar scattering by complex ice particles. This is relatively straightforward to implement in both a radar retrieval algorithm, and the instrument simulators that are now used by the Intergovernmental Panel on Climate Change for evaluating climate models. We find that our new model predicts a significantly higher backscatter than older homogeneous sphere and spheroid models, leading to significantly lower ice water contents being retrieved from a given radar signal. Earlier research funded by this project also found that most ice particles fall with a vertical-to-horizontal aspect ratio of close to 0.6, and their density can be reasonably well described by the "Brown & Francis" density relationship. This is relatively straightforward to implement in a retrieval scheme.

- Another key and original component of the retrieval algorithm is our fast model for radar and lidar multiple scattering; this has been extended to provide the capability to predict the depolarized return due to multiple scattering, using an approximate but sufficiently accurate approach that is computationally very efficient.
Exploitation Route The European Space Agency is planning to use the algorithm for operational production of synergy products when EarthCARE is launched. The intention is to also apply the algorithm to a large volume of data from the existing A-Train of satellites and make the dataset available to the scientific community. The data from the algorithm applied to both EarthCARE and the A-Train is expected to be widely used for evaluating and improving both weather forecast and climate models.

The Adept C++ automatic differentiation software library is available for free at http://www.met.reading.ac.uk/clouds/adept/ - a paper describing the technique is now in print.

The multiple scattering model is available here: http://www.met.reading.ac.uk/clouds/multiscatter/; it has a number of users worldwide, and is being used operationally in production of the CloudSat precipitation product.

Code demonstrating application of the Self Similar Rayleigh Gans Approximation for particle scattering calculations is available here: http://www.met.reading.ac.uk/clouds/ssrga/
Sectors Digital/Communication/Information Technologies (including Software),Environment

URL http://www.met.reading.ac.uk/clouds/adept/
 
Description The Adept automatic differentiation algorithm that we developed and released under an open-source license is of interest for a wide range of numerical applications, not just academic. A mailing list exist to support users of the algorithm and it has around 30 subscribers. The private-sector users include those working in banks, energy companies and medical imagery.
First Year Of Impact 2013
Sector Digital/Communication/Information Technologies (including Software),Energy,Financial Services, and Management Consultancy,Pharmaceuticals and Medical Biotechnology
Impact Types Economic

 
Description Variational Synergy algorithms for EarthCARE (VARSY)
Amount £74,000 (GBP)
Funding ID ESA RFQ AO/1-6823/11/NL/CT 
Organisation European Space Agency 
Sector Public
Country France
Start  
 
Description Variational Synergy algorithms for EarthCARE (VARSY)
Amount £74,000 (GBP)
Funding ID ESA RFQ AO/1-6823/11/NL/CT 
Organisation European Space Agency 
Sector Public
Country France
Start 01/2012 
End 01/2013
 
Title C++ software library "Adept": Automatic differentiation using expression templates 
Description Adept (Automatic Differentiation using Expression Templates) is a software library that enables algorithms written in C++ to be automatically differentiated. This then allows them to be used in optimization problems including satellite retrieval and data assimilation schemes. It uses an operator overloading approach, so very little code modification is required. Differentiation can be performed in forward mode, reverse mode (to compute the adjoint), or the full Jacobian matrix can be computed. Moreover, the novel use of expression templates and several other important optimizations mean that reverse-mode differentiation is 5-9 times faster than the leading libraries that provide equivalent functionality (ADOL-C and CppAD) and less memory is used. In fact, Adept is also typically only around 5-20% slower than an adjoint code you might write by hand, but immeasurably faster in terms of user time; adjoint coding is very time consuming and error-prone. 
Type Of Technology Software 
Year Produced 2012 
Open Source License? Yes  
Impact The software has been used in fields outside meteorology, for example financial modelling, computational fluid dynamics and physical chemistry, as outlined on its Wikipedia page at https://en.wikipedia.org/wiki/Adept_(C++_library). A description of the method has been published in ACM Transactions on Mathematical Software. 
URL http://www.met.reading.ac.uk/clouds/adept/
 
Title CAPTIVATE - cloud, aerosol and precipitation retrieval using a variational technique 
Description This is a retrieval algorithm that estimates the microphysical properties of liquid clouds, ice clouds, aerosols and precipitation by combining active and passive instruments. It is flexible in the instruments that it can use, but typically one would combine cloud radar (optionally with Doppler capability), lidar (optionally with HSRL capability), infrared and shortwave radiances. It has been applied to A-Train satellite data (CloudSat, CALIPSO and MODIS), ground-based data, airborne data and simulated data for the forthcoming EarthCARE satellite. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact This software will be used operationally to process the measurements by the forthcoming EarthCARE satellite.