Data-Driven Methods for a New National Household Energy Survey

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics


We propose a programme of research leading to the establishment of a
new methodology for surveying household energy use nationally, to
complement existing methods (e.g., the English Housing Survey
and regional equivalents), which leverages the rollout of Smart Meters
to achieve cost-effective, detailed understanding of energy use
behaviours. The key enablers of this new methodology are: 1) the
Smart Meters themselves, 2) advances in semi-supervised disaggregation
methods which can infer the behaviours that result in energy use, and
3) other wireless sensors placed in some households to bootstrap the
disaggregation methods. The output will be twofold: a validated
scalable novel survey method suitable for national rollout, providing
significant additional data on energy consumption in UK homes; and 2)
an assessment of which variables can be effectively surveyed with this

Planned Impact

The smart meter rollout represents a £12 billion investment into
energy infrastructure. This proposal seeks to maximise the value to
the government and individual homeowners of that investment. For
government, the research could be of direct benefit in the evaluation
of the smart metering programme. In addition, both DECC and the Smart
Meter Central Delivery Body seek to maximise householder value through
the provision of timely, contextually relevant information about their
energy consuming; this will be enabled by the project's development of
methods for the identification of consumer behaviours from standard
smart meter data. The research will directly contribute to ongoing
DECC efforts to study of energy consumption in UK homes, to provide
the evidence base on which more robust energy demand policies can be
developed and evaluated.

In addition to the development of new national scale survey methods
and analytical techniques for the analysis of smart meter data the
research will directly result in an enhanced understanding of the
spatial density of sensing required in order to cost-effectively
capture data on occupants' energy consuming behaviours. This area of
research is currently seeing substantial commercial growth through the
emergence of a range of SMEs targeting the provision of home energy
management services. Such UK companies are seeking to gain first
mover advantage, via the UK rollout of smart meters, to establish
themselves as global leaders. This research will enable companies to
understand how best to deploy a limited number of sensors within homes
to effectively detect and manage energy consumption.
Description We have developed state of the art methods for disaggregating whole-house electricity data. This will enable much better analysis of energy use in households by appliance type, purely from smart meter data, without the need for surveys.
Exploitation Route Energy service companies might use these methods to provide better information to consumers about their energy use. Policy makers could use them to better understand energy use in households on a national scale.

The state of the art methods for disaggregating whole-house electricity data are being put to use in the social care sector, We are working with a housing association and care provider with their community alarm customers. The disaggregation method is being used in a new project and being combined with machine learning to identify when a vulnerable person maybe in the need of help and to raise an alert.

We are also using the disaggregation methods in current project proposals working with the same housing association and care provider and also with the City of Edinburgh Council (with whom we built a collaboration with in the Enhance Project) in a microgrid and private wire network specifically in the area of demand shifting time of use of household appliances to enable smart use of solar renewable energy and battery storage within the micro grid.
Sectors Communities and Social Services/Policy,Energy,Government, Democracy and Justice

Description Disaggregation methods such as ours are in increasingly widespread use in the energy sector by a variety of players, particularly energy supply companies who see added value in providing a breakdown of household electricity bills to consumers. In May 2021 we published the most extensive and detailed UK household energy dataset (the IDEAL dataset), arising from this and the related IDEAL project: It is described in detail in the Nature Scientific Data paper: In the past 10 months it has been accessed over 3000 times from multiple countries around the world, with a continuing rate of 200-300 accesses per month. As the UK moves its primary domestic heating technology from gas boilers to electric heat pumps, the detailed heating-related data in the IDEAL dataset is helping academic and industry groups to project the impact of this transition on the electricity grid.
First Year Of Impact 2021
Sector Digital/Communication/Information Technologies (including Software),Energy
Impact Types Policy & public services

Description DecarbonISation PAThways for Cooling and Heating (DISPATCH)
Amount £1,401,881 (GBP)
Funding ID EP/V042955/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 08/2021 
End 09/2024
Description The Blackwood Neighbourhood for Independent Living
Amount £12,000,000 (GBP)
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 03/2021 
End 03/2024
Title IDEAL database 
Description Database of detailed data relating to energy use and environmental conditions in 250 households in the Edinburgh region. Households recruited into the study from approximately March 2017 to April 2018. Demographic, attitudinal and some building data. Detailed time series on ambient temperature, humidity, water pipe temperatures, gas use and electricity use. In about 30 households, individual appliance electrical data (for up to 9 appliances), temperature series for individual radiator flow and return, and more temperature sensors associated with particular activities. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact None yet 
Description Article for magazine supplement on reducing carbon emissions 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact We contributed to a supplement to New Statesman entitled Reduce the Carbon, published in October 2015. This articulated in lay terms the potential that demand reduction in the domestic sector can play in the overall energy system, with respect to cost, security and carbon emissions.
Year(s) Of Engagement Activity 2015
Description DataFest18 Smart Energy GB 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact This was a fringe event at the DataFest in Edinburgh at which we talked about about smart meter data and the IDEAL project. The theme of the event was around some of the opportunities that smart energy data presents for innovation in areas such as sustainability and healthcare.
Year(s) Of Engagement Activity 2018