System-wide Probabilistic Energy Forecasting
Lead Research Organisation:
University of Glasgow
Department Name: School of Mathematics & Statistics
Abstract
The UK has binding targets to reduce carbon emission by 80% from 1990 levels by 2050. To achieve this, our energy systems are changing rapidly with a growing portion of electricity coming from renewable energy sources, and electrification of heating and transport. The result of this transition is an electricity system that is increasingly dependent on the weather: as well as having to manage variable amounts of power available from wind and solar resources, demand for electricity is becoming increasingly weather-dependent. Electricity network operators, generators and suppliers must rely on weather forecasts to plan their operations and ensure that supply meets demand, and they must do so in the knowledge that weather forecasts are imperfect, and therefore that future generation and demand uncertain.
This research will develop new energy forecasting methodologies to address the needs of the energy industry in this new paradigm. Energy forecasts are required for all weather-dependent elements of the electricity system, and their uncertainty must be quantified. Critically, there is a high degree of interdependence between uncertainty across the electricity system which must be captured to correctly characterise overall uncertainty. Furthermore, the precise nature of that interdependence will vary depending on specific weather conditions. The methodologies developed here will provide a framework for system-wide energy forecasting considering large-scale meteorological conditions, and provide decision-makers with valuable information about forecast uncertainty.
In addition, specific decision-support tools will be derived to condense voluminous and complex probabilistic forecast information into actionable analytical support. Tools to aid operational decision for power system operators, such as deciding how much back-up power to have available and how to manage constrains on the gird will be developed. Similarly, tools for generators and suppliers will be produced to enable more efficient participation in electricity markets. The overall objective of this work is to reduce the cost, and increase the reliability, of power systems with a high penetration of renewables.
This research will develop new energy forecasting methodologies to address the needs of the energy industry in this new paradigm. Energy forecasts are required for all weather-dependent elements of the electricity system, and their uncertainty must be quantified. Critically, there is a high degree of interdependence between uncertainty across the electricity system which must be captured to correctly characterise overall uncertainty. Furthermore, the precise nature of that interdependence will vary depending on specific weather conditions. The methodologies developed here will provide a framework for system-wide energy forecasting considering large-scale meteorological conditions, and provide decision-makers with valuable information about forecast uncertainty.
In addition, specific decision-support tools will be derived to condense voluminous and complex probabilistic forecast information into actionable analytical support. Tools to aid operational decision for power system operators, such as deciding how much back-up power to have available and how to manage constrains on the gird will be developed. Similarly, tools for generators and suppliers will be produced to enable more efficient participation in electricity markets. The overall objective of this work is to reduce the cost, and increase the reliability, of power systems with a high penetration of renewables.
Planned Impact
The growth of renewables in the UK and around the world is playing a major role in the global effort to mitigate the negative effects of climate change. As a result, power systems must adapt to function in a new paradigm where generation and demand are highly weather dependent, and where operators increasingly rely on weather forecasts, which are inherently imperfect. Equipping decision-makers with detailed forecast and uncertainty information, as this research will do, will enable more economic and reliable power system operation. As such, this work will have significant impact on the transition to low-carbon energy in the UK and globally.
Energy forecasts that quantify uncertainty and tools which convert that information in to actionable decision-support will contribute to continued integration of renewables while maintaining the high level of power system reliability and resilience expected of a developed economy. Furthermore, effective use of accurate uncertainty quantification will result in cost savings from reserve holding and electricity market operation which will translate to savings for energy consumers.
The project will have impact in the short-term resulting from advances in mathematical techniques for large-scale probabilistic forecasting that will have benefits across disciplines, such as econometrics and biostatistics. Over the longer term (3-4 years) the release of forecasting methodologies in an R package will enable researchers and industry to access novel forecasting tools in a timely way. Furthermore, training on use of the package will be provided for project partners and attendees at the final dissemination event. This will serve to up-skill researchers, enhance the effectiveness of users in the energy industry, and lead to new ideas for development and associated R&D funding. This research has already attracted significant interest from National Grid and ScottishPower who are keen to advance the use of probabilistic forecasting in their companies.
Numerous individuals will gain research and professional skills by being part of this research agenda. Both I and my collaborators will acquire skills in global engagement, communication and teamwork while expanding our research portfolios and skills. I will develop skills in meteorological and power system analysis while my collaborators will gain skills in statistical methods and energy domain knowledge. Research students will be up-skilled via interacting with this work through CDT mini-projects and PhDs, as will employees of industrial partners who will receive training in use of the methods produced by this work. This research lends itself to public engagement in understanding the necessity of science and engineering and their relevance to societal issues around energy and climate change. Through public UoS public engagement events and programmes run by the Glasgow Science Centre and Science Festival, which whom I have worked in the past, the importance of engineering research and STEM careers will be promoted.
Energy forecasts that quantify uncertainty and tools which convert that information in to actionable decision-support will contribute to continued integration of renewables while maintaining the high level of power system reliability and resilience expected of a developed economy. Furthermore, effective use of accurate uncertainty quantification will result in cost savings from reserve holding and electricity market operation which will translate to savings for energy consumers.
The project will have impact in the short-term resulting from advances in mathematical techniques for large-scale probabilistic forecasting that will have benefits across disciplines, such as econometrics and biostatistics. Over the longer term (3-4 years) the release of forecasting methodologies in an R package will enable researchers and industry to access novel forecasting tools in a timely way. Furthermore, training on use of the package will be provided for project partners and attendees at the final dissemination event. This will serve to up-skill researchers, enhance the effectiveness of users in the energy industry, and lead to new ideas for development and associated R&D funding. This research has already attracted significant interest from National Grid and ScottishPower who are keen to advance the use of probabilistic forecasting in their companies.
Numerous individuals will gain research and professional skills by being part of this research agenda. Both I and my collaborators will acquire skills in global engagement, communication and teamwork while expanding our research portfolios and skills. I will develop skills in meteorological and power system analysis while my collaborators will gain skills in statistical methods and energy domain knowledge. Research students will be up-skilled via interacting with this work through CDT mini-projects and PhDs, as will employees of industrial partners who will receive training in use of the methods produced by this work. This research lends itself to public engagement in understanding the necessity of science and engineering and their relevance to societal issues around energy and climate change. Through public UoS public engagement events and programmes run by the Glasgow Science Centre and Science Festival, which whom I have worked in the past, the importance of engineering research and STEM careers will be promoted.
Publications
Browell J
(2022)
Predicting Electricity Imbalance Prices and Volumes: Capabilities and Opportunities
in Energies
Browell J
(2022)
Covariance structures for high-dimensional energy forecasting
in Electric Power Systems Research
Farrokhabadi M
(2022)
Day-Ahead Electricity Demand Forecasting Competition: Post-COVID Paradigm
in IEEE Open Access Journal of Power and Energy
Gilbert C
(2023)
Probabilistic load forecasting for the low voltage network: Forecast fusion and daily peaks
in Sustainable Energy, Grids and Networks
Gioia V
(2024)
Additive Covariance Matrix Models: Modeling Regional Electricity Net-Demand in Great Britain
in Journal of the American Statistical Association
Related Projects
| Project Reference | Relationship | Related To | Start | End | Award Value |
|---|---|---|---|---|---|
| EP/R023484/1 | 28/06/2018 | 31/07/2021 | £312,934 | ||
| EP/R023484/2 | Transfer | EP/R023484/1 | 01/08/2021 | 30/07/2022 | £5,040 |
| Description | This work has developed new ways of forecasting Great Britain's electricity supply and demand in the days ahead. Such forecasts are essential to allow generators to schedule power production, and for National Grid to operate the transmission system, ensuring that supply and demand are in balance. In particular, this work has made it possible to accurately quantify the risk of very high or low demand, allowing the electricity market and National Grid to take appropriate action. Compared to current practice, these new methods provide confidence when forecast uncertainty is low, reducing the cost of operating the system, and identify times when risks may be high, ensuring appropriate actions can be taken to maintain reliable supply. |
| Exploitation Route | Outcomes are being taken forwards by the energy industry to improve the way energy supply and demand are forecast, which is expected to support the transition to net-zero at least cost while maintaining high standards of reliability. |
| Sectors | Digital/Communication/Information Technologies (including Software) Energy |
| Description | This project has contributed to multi-million-pound savings to electricity consumers and increase security of energy supply. Research outputs included novel a methodology for quantifying risk in electricity demand forecasts to inform the procurement of "back-up" power by power system operators, and open-source software implementing this and other methods. I designed and undertook two parallel activities that have led to this research having significant impact. First, a innovation project with partners the Smith Institute and TNEI Services project funded by National Grid Electricity System Operator (£400k). This project established the feasibility of implementing method I developed within NGESO and quantified the business and consumer benefits. Secondly, I designed a training course in the use of my open-source software to train potential users in the underlying methods and their implantation. This course has been delivered twice and attended by over 50 individuals from companies including Scottish Power, SSE, National Grid, and TNEI Services, as well as academic researchers. TNEI's engagement is noteworthy as they now use this software in their consulting practice with the wider energy sector. The outcome of these combined activities has been the adoption of a new business process for procuring back-up power at NGESO with an estimated benefit of £75m/year, a saving which is socialised though lower consumer electricity bills in Great Britain. Other benefits include reduced risk of having insufficient back-up power, which is harder to quantify financially but potentially of even greater value to society. Evidenced by investment in implementation of changes to business practice detailed in published reports and National Grid strategy documents, which took place in 2024. Financial benefits expected to be realised from 2025 onwards. |
| First Year Of Impact | 2022 |
| Sector | Energy |
| Impact Types | Economic Policy & public services |
| Description | Energy Forecasting Innovation Conference 2022 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | I organised this 2.5 day conference along with Claudia Neves (another Innovation Fellow) and Bruce Stephen (PI of EPSRC AMIDINE project) to disseminate the outputs of our fellowships. The conference included presentations from ourselves, our industrial partners, and a half-day raining course in the open source tools we have produced. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Training delivered to National Grid ESO |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | I delivered a two-day training course on probabilistic energy forecasting, including practical instruction in the use of open source tools produced by my Innovation Fellowship. These methods and tools are being adopted within the ESO and energy sector more broadly. Similar engagement is being developed with electricity DNOs and gas network operators. |
| Year(s) Of Engagement Activity | 2022 |
