Uncertainty analysis of hierarchical energy systems models: Models versus real energy systems

Lead Research Organisation: Durham University
Department Name: Engineering and Computing Sciences

Abstract

Mathematical models are used widely in the planning and operation of energy systems, and in the development of public energy policy. The aim is to understand the impact of new policies, technologies and market operations. This is particularly significant at present due to the need to decarbonise energy systems over the coming decades, which is driving change in energy supply at a very rapid pace. Specific recent uses of large scale modelling studies in formal government policy Impact Assessments are the 2012 Energy Bill (which among other models uses economic modelling to project future generation investment under different policy options) and the 4th Carbon Budget (which provides the legal limit on GB carbon emissions from 2023 to 2027, and uses the UK MARKAL model for projecting evolution of the energy system under a given background scenario).

This project will study the relationship between mathematical and computer models of energy systems, and the real systems that they attempt to describe, with the purpose of enabling better model-based decisions in industry and government. Until such a relationship has been established between the model and the physical system that the model purports to represent, it is impossible to draw fully robust conclusions based on the model. This model/reality relationship will necessarily be probabilistic, expressing the degree to which uncertainty about the world can be resolved by careful use of the model. We will show, by a careful choice of exemplars, the ways in which such probabilistic relationships may be constructed for a wide range of energy systems models.

When we have linked the model and the physical system, the model can be embedded in a meaningful decision support system for choosing sensible future actions. This involves honest and careful assessment of all of the uncertainties involved in the planning process and consequent forecasting of the uncertainties associated with each possible planning choice. Using the selected exemplars, we will show how to replace current energy planning scenarios with a scrupulous uncertainty based guide to the consequences of current and future actions.

While much of this project involves specific exemplars, our intentions are general, namely to derive methodology which is widely applicable across the whole field of energy systems modelling and planning and therefore to show the potential for transformative analysis across the whole field. For this reason, our exemplars have been chosen to reflect general features common to a wide variety of energy models, which generally comprise systems of interconnected models which are each complex in their own right (e.g. a market model linking to an assessment of the engineering consequences of investment decisions, or a model describing interacting transmission and distribution networks). General methodology for the analysis of computer models will be tailored to the requirements of energy systems analysis. Further, some aspects, and in particular the development of effective uncertainty analysis for linked computer models, will have impact across modelling applications in many other fields.

Exemplars used will include studying the interaction between investment in generating capacity and the risks of supply shortages, the participation of resources embedded within distribution networks in the national energy market, and embedding a model of a particular sector of the energy economy within a model which projects the evolution of the whole energy economy. We will work on these exemplars in discussion with industrial collaborators, in order to identify how our methods must be designed and communicated in order for them to lead to eventual field application.

Planned Impact

The proposed research will benefit:

The government and electricity regulator. Improved systematic methods for analysing uncertainty in complex computer models will enable better public policy decisions to be made based on these models, helping deliver the required carbon reductions with improved cost and security of supply to customers.

The energy supply industry. The methods developed will enable better modelling-based decision making directly on the part of the industry; the Pathways to Impact document lists the various ways in which we will enable this transition of knowledge to the industry.

Energy customers. Ultimately the aim of enabling better model-based decision making by government and the energy industry is to enable the goals of decarbonisation and security of supply to be met at lower cost.

Users of complex computer models across the public, private and third sectors beyond the specific field of energy systems. The innovations in uncertainty analysis made in this project will find application wherever complex structures of linked models are used to support decisions. The methods developed bring potential benefits to any such organisation, resulting in better decision making and ultimately leading to more efficient and effective solutions for their citizens and customers.

The project will also have specific impact within the major industrial projects with which we will collaborate directly on modelling exemplars, notably the statutory Capacity Assessment project (feeding in to more robust design and operation of capacity mechanisms) and National Grid's work on transmission planning.
 
Description We believe this project is the only activity in the world using Bayesian model emulation withing uncertainty quantification in complex energy systems models. In general, investigating uncertainty in the relationship between outputs of a computer model and the real world system it purports to represent requires runnig the model multiple times - however if the model takes a long time to run then only a limited number of runs might be possible. This project has developed new methodology for making maximum use of a small number of runs - this was motivated by the fact that many energy models have very large numbers of inputs and outputs, and the number of runs ideally required increases with the numbers of inputs and outputs of the model.

In the latter part of the project, work has been taken forward through a jointly funded researcher with the UK Centre for Energy Systems Integration. Applications have included building energy modelling and renewable energy system planning - in these topics, papers are either in preparation or review. This work has also led to relevant wider methodology developments, including in scoring rules for probabilistic forecasts.

We have, more generally, demonstrated this class of methods for statistical uncertainty quantification on energy systems models for the first time - this has attracted the interest of many researchers and practitioners during talks and conferences. Future projects are discussed below.
Exploitation Route The first step on this is most likely to be adoption by academic groups with a more applied focus - while we are working with industry and government, and there may be opportunities for consultancy or impact projects on specific questions, the methodology developed might not make that direct leap from methodological academia to industry practice. Following (in autumn 2016) RA Amy Wilson spending 6 weeks on secondment to the government working on implementation of uncertainty quantification, we now have an Alan Turing Institute project on translating uncertainty government methods to wide practice, which will hopefully significantly accelerate non-academic impact. In addition, the Centre for Digital Built Britain has taken up thinking arising from this project in its research agenda, following a scoping network which was led by Chris Dent, and the two Investigators have co-written a further related white paper to be published by the Alan Turing Institute.
Sectors Energy,Environment,Government, Democracy and Justice,Transport

 
Description Ideas arising from this project have generated considerable interest in the government analyst community, including discussion with cross-government units who have oversight of modelling practice or who provide recommendations on development of practice. Specific applications of the relevant thinking are starting to develop, including decision analytic work linked to the electricity capacity market, a consulting report to Ofgem on decision analysis, and work with National Audit Office and local authories also connected to the Centre for Energy Systems Integration.
Sector Energy,Government, Democracy and Justice
Impact Types Policy & public services

 
Description IEEE
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Chris Dent has the following roles within IEEE Power and Energy Society: - Vice Chair, Reliability, Risk and Probability Applications Subcommittee - Chair, LOLEWG (which shares experience between practical adequacy studies) - Chair, Task Force on Capacity Value of Solar Power - Chair, Working Group on Review of IEEE Standard 859 (transmission reliability data collection terminology) These contribute to development of industrial and academic knowledge and practice worldwide in the field of power system reliability analysis
 
Description Knowledge Transfer Secondment to BEIS
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact Dr. Amy Wilson (postdoc) seconded to BEIS for 6 weeks to demonstrate prrof of concept implementation on uncertainty quantifiction methods. Result so far is improved knowledge and skills of BEIS analysts.
 
Description Submission to House of Lords Enquiry by Wilson, Goldstein and Dent
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://www.parliament.uk/resilience-of-electricity-infrastructure
 
Description Alan Turing Institute Partnership Project Scheme
Amount £403,000 (GBP)
Organisation Alan Turing Institute 
Sector Academic/University
Country United Kingdom
Start 10/2018 
End 10/2018
 
Description Alan Turing Institute Strategic Priorities Fund
Amount £64,000 (GBP)
Organisation Alan Turing Institute 
Sector Academic/University
Country United Kingdom
Start 11/2018 
End 03/2019
 
Description Centre for Digital Built Britain Research Network
Amount £46,990 (GBP)
Organisation Digital Built Britain 
Sector Private
Country United Kingdom
Start 07/2018 
End 12/2018
 
Description Durham University EPSRC IAA account
Amount £7,610 (GBP)
Organisation Durham University 
Sector Academic/University
Country United Kingdom
Start 08/2016 
End 11/2016
 
Description Linking Whole Energy System Models to the Real World
Amount £250,000 (GBP)
Organisation University of Edinburgh 
Sector Academic/University
Country United Kingdom
Start 06/2020 
End 06/2023
 
Description Local Heat and Energy Efficiency Strategies
Amount £30,000 (GBP)
Organisation Government of Scotland 
Sector Public
Country United Kingdom
Start 10/2021 
End 03/2022
 
Description National Digital Twin Programme CLimate Resilience Demonstrator
Amount £1,000,000 (GBP)
Organisation Department for Business, Energy & Industrial Strategy 
Sector Public
Country United Kingdom
Start 04/2021 
End 03/2022
 
Description DECC/Grid 
Organisation Department of Energy and Climate Change
Country United Kingdom 
Sector Public 
PI Contribution Methodologies for statistical uncertainty quantification
Collaborator Contribution Provided the ddm model as an exemplar
Impact Contributed to numerous presentations and research discussions. In kind contribution is nominal - supply of a model which is used in real public policy decision making has a value which is almost unquantifiable.
Start Year 2015
 
Description DECC/Grid 
Organisation National Grid UK
Country United Kingdom 
Sector Private 
PI Contribution Methodologies for statistical uncertainty quantification
Collaborator Contribution Provided the ddm model as an exemplar
Impact Contributed to numerous presentations and research discussions. In kind contribution is nominal - supply of a model which is used in real public policy decision making has a value which is almost unquantifiable.
Start Year 2015
 
Description ESIPP 
Organisation University College Dublin
Country Ireland 
Sector Academic/University 
PI Contribution Provided letter of support. Providing ongoing input on statistical uncertainty quantification.
Collaborator Contribution Energy system model exemplars. Meteorological data.
Impact Multidisciplinary - engineering, maths, stats, meteorology
Start Year 2014
 
Description Knowledge Transfer Secondment to BEIS 
Organisation Department for Business, Energy & Industrial Strategy
Country United Kingdom 
Sector Public 
PI Contribution Knowledge transfer secondment of postdoc research Amy Wilson to BEIS, to implement proof of concept implemetation of methodology developed in academic work, and to scope future opportunities for collaboration.
Collaborator Contribution £750 cash for travel to meetings. Provision of runs from one of their computer models to use as exemplar in academic research. In-kind support in terms of time during the secondment.
Impact Presentations at various events, including tutorial day on energy systems planning under uncertainty.
Start Year 2015
 
Description iiESI 
Organisation European Energy Research Alliance (EERA)
Country Belgium 
Sector Academic/University 
PI Contribution Input to scoping documents of field of energy systems integration - EERA Joint Programme scheme of work, and International Institute for Energy Systems Integration green paper and other reports
Collaborator Contribution Overall coordination of activity in ESI, further intellectual input to reports, event organisation
Impact All disciplines relating to energy systems are involved. EERA JP in ESI scheme of work
Start Year 2014
 
Description iiESI 
Organisation Science and Technologies Facilities Council (STFC)
Department ISIS Neutron and Muon Source
Country United Kingdom 
Sector Academic/University 
PI Contribution Input to scoping documents of field of energy systems integration - EERA Joint Programme scheme of work, and International Institute for Energy Systems Integration green paper and other reports
Collaborator Contribution Overall coordination of activity in ESI, further intellectual input to reports, event organisation
Impact All disciplines relating to energy systems are involved. EERA JP in ESI scheme of work
Start Year 2014
 
Description Amy Wilson presentation at ISBA 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation at International Society for Bayesian Analysis on statistical model emulation with limited model runs
Year(s) Of Engagement Activity 2018
 
Description Amy Wilson seminars to government 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Seminars to BEIS OR and energy groups
Year(s) Of Engagement Activity 2016
 
Description BEIS Digital Twin steering group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Member of advisory group for a Business, Energy and Industrial Strategy-funded pilot project at Energy Systems Catapult
Year(s) Of Engagement Activity 2021,2022
 
Description Blog article on energy modelling 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Blog article on good modelling practice, as part of a University of Edinburgh series relating to the Scottish Government's energy strategy
Year(s) Of Engagement Activity 2017
URL http://blogs.sps.ed.ac.uk/global-environment-society-academy/2017/05/22/energy-systems-modelling-mod...
 
Description Chris Dent and Amy Wilson presentations on capacity markets and uncertainty quantification at the Isaac Newton Institute Mathematics of Energy Systems programme 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Presentations to mixed audience of researchers, industry and policy - part of origin of subsequent consultancy work on decision support for energy network planning.
Year(s) Of Engagement Activity 2019
 
Description Chris Dent tutorial talk at N American Electricity Reliability Corporation Probabilistic Analysis Forum 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited talk at major N American workshop on resource adequacy assessment
Year(s) Of Engagement Activity 2019
 
Description GO Science masterclass 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Invited "Masterclass" by Chris Dent on modelling in public policy to the Government Office of Science Data Science Stream
Year(s) Of Engagement Activity 2018
 
Description ICMS Energy Systems Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Scoping workshop on mathematical science research needs in management of energy systems. Linked to Isaac Newton Institute programme on Mathematics of Energy Systems in 2019.
Year(s) Of Engagement Activity 2018
URL http://www.icms.org.uk/energynetworks.php
 
Description Industry workshop linked to Isaac Newton Institute proposal 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Co-organised industry scoping session associated with a proposal for an Energy Systems programme at the Isaac Newton Institute for Mathematical Sciences.
Year(s) Of Engagement Activity Pre-2006,2017
URL https://iaciac.github.io/lobanet/events/
 
Description Invited presentations in 2020-21 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited presentations to
- International Conference on Probabilistic Methods Applied to Power Systems
- Energy Regulators Regional Association
- Cambridge Society of Edinburgh
- European Union Agency for the Cooperation of Energy Regulators
Year(s) Of Engagement Activity 2020
 
Description Invited presentations to UK industrial mathematics community 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Dr Chris Dent has given invited presentations at:
- Turing Gateway to Mathematics event on "Maths and Public Policy - Cities & Infrastructure", see http://www.turing-gateway.cam.ac.uk/mpci_mar2015-programme
- KTN Industrial Mathematics community event, to stimulate interest in links between the mathematical sciences community and energy systems applications
- KTN "Mathematics in Energy Systems" workshop, see https://www.eventbrite.co.uk/e/energy-catalyst-brokerage-event-london-tickets-19192257559?aff=erelexporg

This outreach activity drew together experience from a number of different EPSRC projects.
Year(s) Of Engagement Activity 2015,2016
 
Description KTN Energy Study Group, Edinburgh, December 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Brainstorming session between mathematical scientists and five industry/government challenges
Year(s) Of Engagement Activity 2009,2018
URL http://www.icms.org.uk/KTN_Energy_SG.php
 
Description Schools outreach by Amy Wilson 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact - Speaker at UK mathematics trust maths circle event, May 2015, Hutcheson's school Glasgow.
- International Women in Engineering day, 23/6/2016 - went to Whitworth Park school in Durham to talk to students about being a female in STEM.
Year(s) Of Engagement Activity 2015,2016
 
Description Seminar and research discussions at NREL 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation and research discussions at National Renewable Energy Laboratory in Denver, CO, developing common interests in resource adequacy assessment and energy system modelling. Particular topics under energy system modelling are the incorporation of operational constraints into planning models, and uncertainty quantification in plannign models, linking to the Future Conventional Power project.
Year(s) Of Engagement Activity 2019
 
Description Seminar at Ofgem on capacity markets and decision support, by Chris Dent, Amy Wilson and Stan Zachary 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Seminar at energy regulator. Part of series of events leading to consultancy on decision support for network planning.
Year(s) Of Engagement Activity 2019
 
Description Tutorial at MNIT Jaipur and seminar at IIT Delhi 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Gave tutorial on reliability analysis the day before HEAPD project workshop at MNIT Jaipur, and seminar on same subject at IIT Delhi the day after the HEAPD workshop.
Year(s) Of Engagement Activity 2016
 
Description Tutorial day on Energy System Planning Under Uncertainty 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Tutorial day for academics and practitioners. Around 40 delegates including speakers. Aim was to translate methods from mathematical science to application disciplines. Slides and videos of talks available on website, see URL.
Sponsored by Hubnet and CESI. Other grants listed indicate source of methodology presented in one or more talks, or payment of travel for a speaker.
Year(s) Of Engagement Activity 2016
URL http://icms.org.uk/workshops/energytutorialday
 
Description Visit to industry colleagues in California to broaden impact links 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Visit to colleagues in California, including CA Independent System Operator, Public Utilities Commission, Stanford Research International, Electric Power Research Institute. Aim is to broaden impact of EPSRC work on uncertainty quantification in complex computer models, including discussion of future collaboration and industry projects.
Year(s) Of Engagement Activity 2017
 
Description Workshop on "Modelling in public policy" 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Scoping workshop on research needs in modelling to support public policy, report at http://hubnet.org.uk/filebyid/1130/Modelling_in_Public_Policy.pdf
Year(s) Of Engagement Activity 2017