Extracting likely scenarios from high resolution ensemble forecasts in real-time

Lead Research Organisation: University of Reading
Department Name: Meteorology

Abstract

National weather forecast centres are moving to a new generation of ensemble forecast systems that run multiple
convection-permitting model forecasts (grid spacing ~2km). They are needed because they can partially resolve the dynamics of small-scale high impact weather phenomena such as intense precipitation. This approach provides a large number of detailed forecasts of local weather for any given forecast time and location, and hence a wealth of new information about the predictability of high-impact events that affect society such as destructive winds, flash flooding, snow and fog. Since the Met Office MOGREPS-UK forecast was one of the first of such systems to go operational, there is now a unique 5-year forecast dataset. One barrier to the full and effective use of these new forecasts is that it is considerably more difficult for a human to fully process such a large amount of information in time to communicate early warnings. Therefore, there is a pressing need to develop a capability to synthesise these data into a manageable number of plausible scenarios or storylines that capture the phenomena of concern and provide emergency responders a clear understanding of the possible outcomes they may face together with an estimate of risk.

The project aims to develop new techniques for extracting clusters (groups of similar forecasts) from the ensemble which can be used to provide forecasters a small set of possible scenarios that can be readily understood by end users. Three approaches will be explored in generating scenarios from ensembles: a top-down approach from clustering global and regional forecast ensembles together, a bottom-up approach from statistical clustering of weather variables at high resolution and an approach using physical insight to partition an ensemble before statistical matching. Case studies will be performed that include situations in which the ensemble is perceived to have produced insufficient variability in outcomes (such as a case in winter 2017 in which all MOGREPS-UK ensemble members produced too much snow over southern England). The goal is to improve early warning services and risk-based decision making.
Training opportunities:
Through collaboration with the Met Office, you will have the opportunity to work with researchers in high resolution modelling and forecast evaluation, operational forecasters and the multi-disciplinary team with expertise in hazards and communication with emergency responders.
While on placement in the Met Office headquarters (Exeter) in an operational research environment the student will experience the challenges of real-time forecasting, the process of upgrading the operational numerical models and production of user-facing forecast products. They will gain an appreciation of user-needs and the degree to which added value to forecasts can be created by understanding the user problems and the decisions they need to make.
At Reading, you will be part of the Dynamical Processes Research Group in the Department of Meteorology which brings together about 40 researchers working on the dynamics of weather systems and climate in weekly group meetings. Together with the Mesoscale Dynamics and Data Assimilation Research Groups, this forms a world-renowned critical mass of atmospheric dynamics and predictability research and would enable you to develop your research with help from the other researchers in the group.
Student profile:
This project would be suitable for students with a degree in physics, mathematics or a closely related environmental or physical science. Experience of computational statistics and some prior knowledge of programming in python, matlab or similar would be desirable. Empathy with users of forecasts and understanding of the needs of professional forecasters is an important aspect of the project.
Funding particulars:
This project has CASE sponsorship from the Met Office in addition to the NERC studentship funding.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
NE/R008590/1 01/10/2018 30/09/2022
2109529 Studentship NE/R008590/1 01/10/2018 30/09/2022 Kristine Boykin
 
Description The Met Office's unified model produces a large number of forecasts for forecasters to examine multiple times a day. When high impact weather occurs, such as wind storms, flood inducing rainfall, etc., time is critical and forecasters must produce both accurate and timely forecasts and warnings to users. By applying the novel clustering method I've developed, several forecasts can be reduced to a few key scenarios that forecasters can then examine and use to determine the probability of each scenario. This method will improve the use of ensemble systems for forecasting by reducing the complexity of data forecasters must visually process and reducing the time required to do so.
Exploitation Route The method I've developed may go into direct use at the Met Office and potentially be applied by other forecasting models in the future.
Sectors Environment

 
Title A novel clustering technique for ensemble weather forecasts for high-impact events 
Description I developed a novel clustering technique for use on ensemble forecasts in real-time to provide operational meteorologists potential weather scenarios by reducing the amount of data they must go through before issuing a forecast. This technique combines k-medoid clustering and the fractions skill score to compare members spatially at each time interval across a forecast, compares clusters temporally, then optimizes the number of clusters based on when clustering reaches a set level of distinction, and finally extracts representative members as potential scenarios for operational meteorologists. 
Type Of Material Improvements to research infrastructure 
Year Produced 2021 
Provided To Others? No  
Impact The method is currently running on the Met Office UK research system and is providing products regularly. It is unofficially available for use by operational meteorologists via an internal website. It has generated a great deal of interest as an upcoming product, particularly for medium-range forecasting, showing the benefits of clustering on even smaller ensembles. 
 
Description Met Office 
Organisation Meteorological Office UK
Country United Kingdom 
Sector Academic/University 
PI Contribution The novel method I am developing will be directly used by the Met Office forecasters on forecasts produced by the unified model to improve ensemble forecasting.
Collaborator Contribution Met Office is supplying data that is used in the development of the method and will in turn be implementing the method once it is complete.
Impact The method is currently being integrated into the Met Office system for further development. Outputs and outcomes are forthcoming.
Start Year 2018
 
Description Met Office CASE sponsorship 
Organisation Meteorological Office UK
Country United Kingdom 
Sector Academic/University 
PI Contribution The studentship had a CASE partnership and so the research was relevant to the strategic science interests of the Met Office. In particular the goal to encourage operational meteorologists to make better use of ensemble forecast information in creating their forecasts and advisories.
Collaborator Contribution Two Met Office researchers were co-supervisors and attended weekly supervisory meetings (online). The Met Office hosted placements at their headquarters in Exeter on a number of occasions - once it was possible following the COVID restrictions.
Impact A report on the winter forecast testbed activity run as part of this studentship project. Papers in preparation on the research.
Start Year 2019
 
Title Implementation of a novel clustering method for ensemble weather forecasts 
Description The method reduces multiple weather forecasts by clustering then selects a series of representative forecasts and produces plots for use in meteorological forecasting of medium-range weather, including potential high-impact scenarios. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2021 
Impact Products from the method, being currently run internally on Met Office research systems, are regularly available for operational meteorologists to use for upcoming forecasting, reducing the time required to examine an ensemble's worth of data before issuing a forecast. It was used in the Met Office Winter 2022 Testbed and is under further development. 
 
Description Atmospheric Science Conference 2021 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This conference focused on atmospheric science solutions and impacts to various environmental issues such as climate change and natural disasters.
Year(s) Of Engagement Activity 2021
URL https://www.atmosphericscienceconference.uk/
 
Description HIWeather 2020 Workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The HIWeather workshop presents current research and core topics concerning the improvement of weather-related warnings. Seminars and workshops were held on communication, warnings, and hazards. This event is a great way for people across the weather related spectrum, from forecasters and emergency managers to researchers and industry, to learn about the latest high-impact weather related research and the improvements being implemented around the world.
Year(s) Of Engagement Activity 2020
URL http://hiweather.net/article/18/1.html
 
Description Joint DTP Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact A joint conference of various DTPs where DTP graduate students present their work to their other current and incoming DTP graduates students. The conference both displays the variety of fields and projects supported as well as builds community among researchers.
Year(s) Of Engagement Activity 2019
 
Description Met Office Winter Testbed 2022 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The Met Office Winter 2022 testbed brought together different forecasting projects for testing and feedback by professional operational meteorologists and scientists. The aim of the testbed is to improve numerical forecasting outputs with the help of this feedback. The project ran over the course of four weeks in January and February 2022, with the workshops taking place all day. The method I have developed to cluster ensemble forecasts and present potential scenarios to operational meteorologists was one of the sessions that ran every day, each week, and received a great deal of feedback. My role was to introduce and train participants each week to use the products my method produced, facilitate the daily workshops for my method, liaise with an operation meteorologist who prepared briefings on the daily weather to present to testbed participants during my workshop, and help guide the discussion after the exercise. By utilizing a daily survey, I collected data about participants' interpretation and use of the products my method provided. The daily workshop and survey feedback included thoughtful discussion on the merits of how the clustering performed each day, potential ways to refine the process, and further applications my work could be applied to. Participants found merit in my method resulting in an increased interest in its products for future forecasting.
Year(s) Of Engagement Activity 2022
 
Description RMetS Student and Early Career Scientists Conference 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact A conference designed to bring students and early career scientists in meteorology and atmospheric sciences together to both build networks and share research with their peers.
Year(s) Of Engagement Activity 2019
URL https://www.rmets.org/student-conference-19
 
Description RMetS Student and Early Career Scientists Conference 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact A conference designed to bring students and early career scientists in meteorology and atmospheric sciences together to both build networks and share research with their peers.
Year(s) Of Engagement Activity 2020
URL https://www.rmets.org/event/virtual-student-early-career-scientists-conference-2020