DATA: ENABLING HOUSEHOLD ENERGY DATA: INFRASTRUCTURES FOR POLICY AND PLANNING
Lead Research Organisation:
University of Essex
Department Name: UK Data Archive
Abstract
This proposal plans to strengthen data expertise and research partnerships between South Africa and the UK though the formation of a new network aimed at investigating access to and use of household energy data for policy-making and planning. The work planned focuses on data infrastructure and brings together data professionals, energy researchers and policy makers in SA and the UK. It includes both knowledge exchange events and limited capacity building that help us to understand better data needs for answering key research and policy questions, and optimal methods and protocols for governance, storage of and access to these data.
The project's aims are a good fit with the mission of the ODA to deliver outcomes that promote the long-term sustainable growth of South Africa as an emerging knowledge economy. Substantive cases from South Africa highlights some of the challenges faced both by the shortage of data needed to provide evidence for evaluation and impact monitoring of e.g. consumption, fuel poverty and demand reduction, but also the lack of systems and tools to extract and present intelligence from data.
Teams will work across the network to address key policy challenges through shared scoping, planning and analysis of UK and SA household energy data. This proposal connects centres of expertise to further progress in areas where there is less capacity than is needed for progressing the next generation of data infrastructures, and in turn, foster new research agendas. Because the centres involved are recognised as key players in their various existing roles, the fusion of their activities will enhance international pathways to impact.
The project will contribute to the development of an internationally networked cohort of experts charged with brokering of and access to data new and novel forms of data who have the skills to work globally. To meet this final aim, the project will co-design and implement a joint UK-SA workshop programme focused on data management and sharing in the social sciences, aimed at researchers, that contributes to the global data science educational programme being developed under the wing of the Research Data Alliance (RDA). The theme of household energy data will provide an excellent exemplar for the training.
A number of government initiatives in SA and the UK are already exploring the potential of existing big data to support areas of planning and policy development such as resilience, preparedness and response for disasters, and capacity building and so on. Big data presents new opportunities for undertaking data-intensive research through identifying new correlation from the expansive and extant range of available digitally born energy data. Importantly these new data assets need digitally re-structuring to become valuable for answering data-driven research. An extension to the existing data curation and preservation methods is required in order to maximise the value of these new products.
The project's aims are a good fit with the mission of the ODA to deliver outcomes that promote the long-term sustainable growth of South Africa as an emerging knowledge economy. Substantive cases from South Africa highlights some of the challenges faced both by the shortage of data needed to provide evidence for evaluation and impact monitoring of e.g. consumption, fuel poverty and demand reduction, but also the lack of systems and tools to extract and present intelligence from data.
Teams will work across the network to address key policy challenges through shared scoping, planning and analysis of UK and SA household energy data. This proposal connects centres of expertise to further progress in areas where there is less capacity than is needed for progressing the next generation of data infrastructures, and in turn, foster new research agendas. Because the centres involved are recognised as key players in their various existing roles, the fusion of their activities will enhance international pathways to impact.
The project will contribute to the development of an internationally networked cohort of experts charged with brokering of and access to data new and novel forms of data who have the skills to work globally. To meet this final aim, the project will co-design and implement a joint UK-SA workshop programme focused on data management and sharing in the social sciences, aimed at researchers, that contributes to the global data science educational programme being developed under the wing of the Research Data Alliance (RDA). The theme of household energy data will provide an excellent exemplar for the training.
A number of government initiatives in SA and the UK are already exploring the potential of existing big data to support areas of planning and policy development such as resilience, preparedness and response for disasters, and capacity building and so on. Big data presents new opportunities for undertaking data-intensive research through identifying new correlation from the expansive and extant range of available digitally born energy data. Importantly these new data assets need digitally re-structuring to become valuable for answering data-driven research. An extension to the existing data curation and preservation methods is required in order to maximise the value of these new products.
Planned Impact
This proposal plans to strengthen data expertise and research partnerships between South Africa and the UK though the co-investigation of infrastructure for household energy data. This will include knowledge exchange and capacity build events that focus on access to and use of data. Researchers will work strategically across the new network to address key policy challenges by shared analysis of UK and SA household energy data.
Knowledge exchange impacts: Given the multi-disciplinary nature of this project we expect to deliver a wide range of benefits that include publications and conference submissions in social sciences, data informatics, and the energy/natural environmental sciences. For example In liaison with our key stakeholders we plan to host international workshops to exchange big data methodologies for curating household energy data, develop new skills for enabling, using and visualizing this energy data. A number of final reports and case studies will compile findings and investigate future sustainable opportunities and pathways for securing timely access to quality energy data.
Policy making impacts: Increasing amount of accessible and high quality data will facilitate planning and policy-making. Researchers are concerned with (lack of) access to data for data analysis for assessing fuel poverty, consumption and demand reduction. This project will help facilitate provide greater access to a wider network of users facilitated by scoping sources and implementing appropriate governance practices. Part of the programme of work planned is to trial access for the community beyond academia using appropriate safeguards. Since the fruits of securing powerful data resources may not be immediate, and will be an on-going process to build more collections, we expect longer-term impact to manifest itself more effective data-based policy and planning. Case studies will be produced for policy and research consumption.
Research data infrastructure and data profession impacts: The proposal brings together UK and South African data professionals to share expertise and develop new research agendas. It aims to strengthen research in this area of data infrastructure between the UK and South Africa by providing joint activities between existing international centres of excellence that enhance knowledge about how best to enable data for social science and policy research. Through the partnership the project will promote mobility and develop capacity, thus contributing to the development of an internationally networked cohort of experts charged with brokering of and access to data new and novel forms of data who have the skills to work globally. To meet this final aim, the project will co-design and implement a joint UK-SA workshop programme focused on data management and sharing in the social sciences that contributes to the global data science educational programme being developed under the wing of the Research Data Alliance (RDA). The theme of energy data will provide an excellent exemplar for the training,
Engaging users: In addition to the involvement of active energy researchers with on-going or pilot projects to contribute, we are holding knowledge exchange and training and capacity building activities. Within the time frame of this project, four streams of KE and TCB are proposed that aim to up-skill data professional staff and data owners in South Africa. These include: intensive knowledge workshops; a virtual user forum (using a successfully-launched UKDS Google group) to enable consultation and involvement from a wider audience and webinars to showcase projects energy data products and infrastructure solutions and gain feedback.
Communications and engagement: This project will benefit hugely from the PR machine of the UK Data Service that is pro-active in its use of up-to-datewebsite content, blogs, social media outlets and mailing lists.
Knowledge exchange impacts: Given the multi-disciplinary nature of this project we expect to deliver a wide range of benefits that include publications and conference submissions in social sciences, data informatics, and the energy/natural environmental sciences. For example In liaison with our key stakeholders we plan to host international workshops to exchange big data methodologies for curating household energy data, develop new skills for enabling, using and visualizing this energy data. A number of final reports and case studies will compile findings and investigate future sustainable opportunities and pathways for securing timely access to quality energy data.
Policy making impacts: Increasing amount of accessible and high quality data will facilitate planning and policy-making. Researchers are concerned with (lack of) access to data for data analysis for assessing fuel poverty, consumption and demand reduction. This project will help facilitate provide greater access to a wider network of users facilitated by scoping sources and implementing appropriate governance practices. Part of the programme of work planned is to trial access for the community beyond academia using appropriate safeguards. Since the fruits of securing powerful data resources may not be immediate, and will be an on-going process to build more collections, we expect longer-term impact to manifest itself more effective data-based policy and planning. Case studies will be produced for policy and research consumption.
Research data infrastructure and data profession impacts: The proposal brings together UK and South African data professionals to share expertise and develop new research agendas. It aims to strengthen research in this area of data infrastructure between the UK and South Africa by providing joint activities between existing international centres of excellence that enhance knowledge about how best to enable data for social science and policy research. Through the partnership the project will promote mobility and develop capacity, thus contributing to the development of an internationally networked cohort of experts charged with brokering of and access to data new and novel forms of data who have the skills to work globally. To meet this final aim, the project will co-design and implement a joint UK-SA workshop programme focused on data management and sharing in the social sciences that contributes to the global data science educational programme being developed under the wing of the Research Data Alliance (RDA). The theme of energy data will provide an excellent exemplar for the training,
Engaging users: In addition to the involvement of active energy researchers with on-going or pilot projects to contribute, we are holding knowledge exchange and training and capacity building activities. Within the time frame of this project, four streams of KE and TCB are proposed that aim to up-skill data professional staff and data owners in South Africa. These include: intensive knowledge workshops; a virtual user forum (using a successfully-launched UKDS Google group) to enable consultation and involvement from a wider audience and webinars to showcase projects energy data products and infrastructure solutions and gain feedback.
Communications and engagement: This project will benefit hugely from the PR machine of the UK Data Service that is pro-active in its use of up-to-datewebsite content, blogs, social media outlets and mailing lists.
Organisations
People |
ORCID iD |
| Louise Corti (Principal Investigator) |
Publications
Corti L
(2017)
Smarter Household Energy Data: blog
Corti, L.
(2017)
Delving into Data: Building data infrastructure
Harris T
(2017)
Aiming for a Moving Target: The Dynamics of Household Electricity Connections in a Developing Context
in World Development
Wittenberg, M.
(2017)
Decomposing changes in household measures: Household size and services in South Africa
in Demographic Research
| Description | Collaboration on research data infrastructure is increasingly becoming the norm as global partnerships between research institutes grow. One example of where this has delivered successful outcomes is the partnership between the UK and South Africa on the knowledge exchange project 'Smarter Household Energy Data: Infrastructures for Policy and Planning'. This recently completed project resulted in collaboration on social science data services and the use of applied research to answer policy-relevant questions in the household energy domain. Knowing more about domestic energy consumption can better inform strategies to deal with fuel demand and poverty, and strategies for energy savings and efficiency. The 18-month project, funded by the Economic and Social Research Council (ESRC) in the UK and the National Research Foundation (NRF), strengthened data expertise and research partnerships between South Africa and the UK, though the formation of a new network created to investigate access to, and the use of, household energy data for policymaking and planning. The work focused on data infrastructure and brought together data professionals, energy researchers and policymakers across the two countries. The two lead partners in the partnership through this period were two leading social science data archives: the UK's flagship data service, UK Data Service, curator of the largest collection of digital data in the social sciences and humanities; and South Africa's premier data service, DataFirst at the University of Cape Town (UCL), which archives and provides online research access to African microdata. The interdisciplinary nature of the project invited key researcher centres in to learn from each other's methodological, analytic and technical approaches, and to collectively identify and increase the amount of accessible and high-quality data for energy research and research data infrastructure. The project ran a number of knowledge workshops that helped understand data needs for answering key research and policy questions, as well as agreeing on optimal methods and protocols for governance, storage of, and access to, these data. It also developed and delivered two very successful capacity building impacts through dedicated week-long summer schools in both countries, which gave further impetus to the foundation of principles for the relatively young field of data science'. In the UK enhanced a data assets registry of energy data sources, worked on methods and tools for assessing smart meter data quality, and span off a significant new funded project with energy researchers to build a Smart Meter Data Portal, which has data governance, usability and technical challenges that push the boundaries of current research practices. In South Africa some key long running large scale data sources were restructured and linked with other data sources e.g. satellite to create new knowledge about the economic impact of rural electrification. Findings suggested that despite significant progress over the last 20 years, basic access to electricity in South Africa is by no means stable or guaranteed, and remains one of the largest development issues faced by post-apartheid South Africa. Research helped shed a spotlight on some of the reasons for inequality in public service delivery and access to electricity. |
| Exploitation Route | We hope that this project has far-reaching benefits. Given the multidisciplinary nature of this project, some of the outcomes can be taken up and exploited by others. All of the progress has been reported both informally in our blog, with a non-academic audience in mind. Outcomes that can be taken up in South Africa and in the UK by research-driven players - academic, policy and industry - include shared content from: the summer schools, data needs and data quality workshops; data management workshops; curated datasets; published case studies setting out substantive, methodological and technical issues arising from the collaborative project; and finally from the technical and tools development work being undertaken by the UK in building a smart meter research portal. As a specific example, in liaison with our key stakeholders we hosted workshops in both South Africa and the UK to exchange methodologies for evaluating, curating and hosting household energy data, and to develop new skills for enabling, using and visualising these data. Ultimately we hope that the work will help in the delivery of outcomes that promote the long-term sustainable growth of South Africa as an emerging knowledge economy. |
| Sectors | Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Energy,Government, Democracy and Justice,Culture, Heritage, Museums and Collections,Security and Diplomacy |
| URL | https://www.ukdataservice.ac.uk/about-us/our-rd/smarter-household-energy-data |
| Description | his project ran from 2015-17 and was funded under the ESRC/NRF International Centre Partnership. The UK Data Service worked with South Africa's DataFirst on knowledge transfer on big data infrastructure - across data, technical and capacity building angles. This 18-month project strengthened data expertise and research partnerships between South Africa and the UK, through the formation of a new network created to investigate access to, and the use of, household energy data for policymaking and planning. The work focused on data infrastructure and brought together data professionals, energy researchers and policymakers in SA and the UK. It made use of knowledge exchange and capacity building events that helped us to better understand data needs for answering key research and policy questions, as well as agreeing on optimal methods and protocols for governance, storage of, and access to, these data. Teams worked across the network to address key policy challenges through shared scoping, planning and analysis of UK and South African household energy data. Substantive cases from the UK focused on how smart meter data could be harmessed to address end user demand and fuel poverty. Case studies in South Africa highlighted some of the challenges faced both by the shortage of data that is needed to provide evidence for evaluation and impact monitoring of, for example, consumption, fuel poverty and demand reduction, but also the lack of systems and tools to extract and present intelligence from data. The work connected centres of expertise, beyond academia, to further progress in areas where there is less capacity than is required for progressing the next generation of data infrastructures, and in turn, foster new research agendas. For the UK Data Service, a key aim was also to develop and test infrastructure for 'big data' with household energy data as a limited yet realistic use case. The project further aimed to contribute to the development of an internationally-networked cohort of experts charged with the brokering of, and access to, new and novel forms of data who have the skills to work globally. To meet this aim, the project designed a joint UK-SA five day summer school programme that focused on data management and sharing in the social sciences, aimed at researchers. The theme of household energy data provided an excellent exemplar for the training. The summer school built on the global data science educational programme developed under the wing of the Research Data Alliance (RDA). Our project led to collaboration around designing, building and making optimal use of cutting-edge big data technologies, in both the UK and in South Africa, with both government and local government departments and organisations that support domestic energy customers, such as the Energy Savings Trust in the UK. The interest is in using data-driven models using new data platforms to encourage efficient policy making in the energy sector. In South Africa the big data methods demonstrated using satellite nightlight data were found to be of great interest to the National Association of Astronomers who are interested in demonstrating social and economic benefits of astronomy data and methods. The adoption of methods on how to quality assess and share datasets for energy policy has also been taken up by those who hold key data sources. |
| First Year Of Impact | 2017 |
| Sector | Digital/Communication/Information Technologies (including Software),Environment |
| Impact Types | Societal,Economic,Policy & public services |
| Description | ESRC NCRM Collaborative Projects |
| Amount | £80,000 (GBP) |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2018 |
| End | 12/2018 |
| Description | Smart Meter Research Portal (SMRP) |
| Amount | £6,000,000 (GBP) |
| Funding ID | EP/P032761/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2017 |
| End | 09/2022 |
| Title | Data Service as a Platform (DSaaP) |
| Description | The Data Service as a Platform (DSaaP) is an online digital platform that will enable you to explore and analyse more high quality social and economic data, anytime, anywhere, in a safe and trusted environment. Free at the point of access to all academics and non-commercial researchers, the open source platform will be fully available by the end of 2019. The infrastructure is jointly funded by the Economic and Social Research Council, and the Engineering and Physical Sciences Research Council, and is developed by the UK Data Service at the University of Essex. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Impact | Winning of a large award from EPSRC to build a Smart Meter Data Research Portal (SMRP) and interest from Amazon Web Services and Cloudwick on the application of Hadoop technology in academia |
| URL | http://dsaap.info |
| Title | UK Energy data asset register |
| Description | This is register of data available in the UK for undertaking household energy research |
| Type Of Material | Database/Collection of data |
| Year Produced | 2017 |
| Provided To Others? | Yes |
| Impact | This register has been updated and collates all the most significant data sources available. It has been accessed by researchers around the UK. |
| URL | http://reshare.ukdataservice.ac.uk/853019/ |
| Description | CODATA-RDA Working Group on Research Data Science Summer Schools |
| Organisation | Research Data Alliance (RDA) |
| Country | Global |
| Sector | Charity/Non Profit |
| PI Contribution | We will contribute a summer school course in Cape Town specifically for social scientists to build on the core courses that are being run by the CoData-RDA collaboration. |
| Collaborator Contribution | The CoData-RDA collaboration have set up and are running .a first full introductory course on data science will take place in August 2016 at the Abdus Salam International Centre for Theoretical Physics in Trieste, Italy. They have secured sponsorship from the ICTP and from The World Academy of Science, as well as CODATA and RDA. the course aims to accommodate up to 120 students at the start of their post graduate careers and offers travel bursaries 30-40 students. |
| Impact | Collaboration on development and running of summer school courses on data science |
| Start Year | 2015 |
| Description | CODATA-RDA Working Group on Research Data Science Summer Schools |
| Organisation | The Committee on Data for Science and Technology (CODATA) |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | We will contribute a summer school course in Cape Town specifically for social scientists to build on the core courses that are being run by the CoData-RDA collaboration. |
| Collaborator Contribution | The CoData-RDA collaboration have set up and are running .a first full introductory course on data science will take place in August 2016 at the Abdus Salam International Centre for Theoretical Physics in Trieste, Italy. They have secured sponsorship from the ICTP and from The World Academy of Science, as well as CODATA and RDA. the course aims to accommodate up to 120 students at the start of their post graduate careers and offers travel bursaries 30-40 students. |
| Impact | Collaboration on development and running of summer school courses on data science |
| Start Year | 2015 |
| Description | Partnerhsip with RCUK Centre for Environmental Epidemiology, University College London |
| Organisation | University College London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Partnership on the household energy research demonstrators for big data environment. We bring data acquisition, curation and research access expertise |
| Collaborator Contribution | They bring domain expertise and access to high quality research in the household energy data domain. |
| Impact | Ongoing scoping of research infrastructure for UK smart meter data |
| Start Year | 2015 |
| Description | Partnership with ICSSR Indian Data Service (May 2016 - Still Active) |
| Organisation | Information and Library Network |
| Country | India |
| Sector | Public |
| PI Contribution | Making data available for research is an ongoing challenge in India, with open access an exception; government departments cautious to grant access to data; and private sector data, wherever accessible, expensive to make available for research. A major driver has been the development of the Indian Department of Science and Technology publication of its National Data Sharing and Accessibility Policy (NDSAP) for non-sensitive data (2012). The ICSSR's remit in the NASDAP included the development of and support for documentation services and the supply of data, through the development of a repository of Social, Economic and Political data on India, with data processing and analytics tools to support researchers, teachers and policymakers who rely on high-quality social and economic data for their research. A road map and implementation plan for ICSSR's data service were produced in 2014, outlining the proposed information architecture for the data repository, setting out key components into data acquisition, data pre-processing and analytics, metadata, software requirements and technology; providing an excellent starting point from which to develop a data service for India. Senior representatives of the ICSSR and other organisations with an interest in the implementation of the ICSSR Data Service visited the UK Data Archive at the University of Essex, a partner in the UK Data Service, in January 2015, supplemented by a two-day training workshop. Over the two days our colleagues from India learned about the data service infrastructure and funding environment in the UK, and how our own data service is managed and run including the data ingest pipeline, from negotiating with data owners and licensing, to processing and documenting data, though to providing access to and user support for data users. Following the visit UKDA prepared a report setting out some practical observations that would help put the plans for the ICSSR Data Service into action. Key areas included the need to factor in continuous appraisal and monitoring of the collection, the (possible) deaccessioning of collections, dealing with personal data, disaster recovery, the relationship to other workflows, for example, the research funding application process (as in the ESRC's Research Data Policy) and ensuring adequate data citation mechanisms, each of which has implications for establishing information architecture and the costs involved. Our advice to anyone wishing to establish a new data repository is to frame key activities using the Open Archival Information System (OAIS) (ISO 16363) as a guide, a standard which provides an intuitive reference model for digital repositories, especially to help structure workflows, staffing and IT operations. |
| Collaborator Contribution | In 2016 following the visit of ICSSR in 2016 they launched a new national data service using the same technology as the South African partners. the team in India have built some open source data visualisation tools that will be of benefit to both the UK and the South Africa. Collaboration continues. |
| Impact | New data archive set up in India, using the same technologeis as South Afica. Interest in scaling up for big data as with the South Africa model Training session on managing and sharing qualitative data provided for Indian Data Service, March 2017 |
| Start Year | 2016 |
| Description | partnership with DataFirst on energy data infrastructure |
| Organisation | University of Cape Town |
| Country | South Africa |
| Sector | Academic/University |
| PI Contribution | Fruitful collaboration on scoping household energy data needs and research project that might benefit from big data infrastructure. we bring the etch and data infrastructure expertise |
| Collaborator Contribution | DataFirst bring high quality research use cases |
| Impact | None yet as only 4 months in. |
| Start Year | 2015 |
| Description | 4 Day Summer school: Encounters with Big Data: An Introduction to using Big Data in the Social Sciences |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The highly successful week long course , 'Encounters with Big Data: An Introduction to using Big Data in the Social Sciences' was rerun as part of the University of Essex Institute of Analytics and Data Science in the UK in July 2018. The foundation of the course lies in one of the key objective of the Smarter Household Energy Data project, a joint International Centre Partnership Grant between the UK Data Service and DataFirst and funded by the Economic and Social Research Council in the UK and the National Research Foundation in South Africa, to create a collaborative research infrastructure for large-scale household energy data. 25 researchers, from across sectors and countries, attended the school (for which bookings were twice filled). Participants fed back that they had gained great support in understanding and analysing large and complex datasets, focusing on using the power of popular statistical software like R in a big data environment. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.ukdataservice.ac.uk/news-and-events/eventsitem/?id=5349 |
| Description | 5 Day Summer school: Encounters with Big Data: An Introduction to using Big Data in the Social Sciences |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The highly successful 5 day course , 'Encounters with Big Data: An Introduction to using Big Data in the Social Sciences' was rerun in the UK in February in August 2017. The course met one of the objectives of the Smarter Household Energy Data project, a joint International Centre Partnership Grant between the UK Data Service and DataFirst and funded by the Economic and Social Research Council in the UK and the National Research Foundation in South Africa, to create a collaborative research infrastructure for large-scale household energy data. 25 researchers, from across sectors and countries, attended the school (for which bookings were twice filled). Participants agreed that they felt they had gained great support in understanding and analysing large and complex datasets, focusing on using the power of popular statistical software like R in a big data environment. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.ukdataservice.ac.uk/media/604994/ukds-case-studies-upskilling.pdf |
| Description | Big data: strategies for understanding your big data and getting new knowledge from it, August 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nathan Cunningham delivered a half-day seminar at UCT on How to understand Big Data: Strategies for understanding your big data and getting new knowledge from it, with UCT staff in attendance from across the research and IT areas. This event was very much pitched at an introductory level to big data thinking, analysis, and tools. Nathan addressed the 4 Vs, noting that while Volume, Velocity and Variety were all indicative about the shape of big data you might be dealing with, the most important V of big data is its Value. We need new strategies for achieving this value for scientific research, and Nathan finished with showcasing the Hadoop Open Data Platform currently being set up to handle Big Data at the UK Data Service. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/11/10/uk-visit-to-south-africa-research-site... |
| Description | Data Quality Workshop, July 2017, Cape Town |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | DataFirst and the UK Data Service hosted a workshop to discuss data quality issues in South African national surveys. The objective of this workshop was to share information about what we know and don't know about progress on meeting the developmental challenges of South Africa since the end of apartheid. This event focused on innovative measurement approaches. DataFirst and the UK Data service are working together in an international partnership, funded by the ESRC/NRF, looking at "Big Data" in the context of household energy. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2017/08/15/cape-town-data-quality-workshop-measur... |
| Description | Energy researcher meeting held at the UCL Energy Institute, June 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | On June 24 2016, a workshop was held as part of our ESRC-NRF International Centre Partnership grant between UK Data Archive and DataFirst, with a view to creating a collaborative research infrastructure for household energy data. A productive workshop was held in Cape Town in January 2016 to discuss active research projects that make use of South African energy data. The present workshop was a small roundtable meeting made up three of our colleagues from DataFirst and UK researchers who have expressed an interest in being early adopters of the system. The meeting aimed to elicit use cases from researchers. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/10/06/uk-energy-research-exemplar-projects-s... |
| Description | Household Energy Workshop, UCT, South Africa |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | This meeting was the project's first engagement event held in Cape Town on 27 January 2016. It brought together South Africa based energy specialists, statisticians and data scientists from academia, industry, the third sector, and government, including researchers from the UCT Energy Research Centre and the National Statistics Institute. The aim was to scope data needs and challenges, and relevant use cases for the project. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/ |
| Description | How to understand Big Data: Strategies for understanding your big data and getting new knowledge from it, Sept 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Nathan Cunningham delivered a half-day seminar at the WITS Rural site staff on How to understand Big Data: Strategies for understanding your big data and getting new knowledge from it. This event was very much pitched at an introductory level to big data thinking, analysis, and tools. Nathan addressed the 4 Vs, noting that while Volume, Velocity and Variety were all indicative about the shape of big data you might be dealing with, the most important V of big data is its Value. We need new strategies for achieving this value for scientific research, and Nathan finished with showcasing the Hadoop Open Data Platform currently being set up to handle Big Data at the UK Data Service. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/11/ |
| Description | Knowledge exchange symposium: 21st Century Data Infrastructure for Research, Cape Town Sept 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | A joint symposium was held on 11 July 2017 at the University of Cape Town (UCT) organised by the eResearch at UCT, Data First and the UK Data Service. The meeting aimed to discuss and encourage debate around the requirements necessary to scale data services and data infrastructure for research. Hosted by Dr Dale Peters, Director of UCT eResearch, the symposium comprised of presentations from Louise Corti and Nathan Cunningham of the UK Data Service, Martin Wittenberg, Director of DataFirst. Anwar Vahed of the Data Intensive Research Initiative of South Africa (DIRISA), Russ Taylor of the Department of Astronomy at UCT (IDIA) and Rob Simmonds of the Department of Computer Science, UCT. Attendees of the symposium hailed from a range of disciplines including data science, bioinformatics and economics. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2017/08/15/knowledge-exchange-symposium-21st-cent... |
| Description | Lecture: Wellcome Africa and Asia Data Managers Group, Sept 2017 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This meeting, organised by the Wellcome International Organisations and Partnerships division, offered the opportunity for the IT and Data Managers of Wellcome funded programmes and other scientific organisations to meet each other and provide a forum for knowledge exchange. Louise spoke about the UK-ZA energy project and on preparing longitudinal population study data for sharing. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.ukdataservice.ac.uk/news-and-events/eventsitem/?id=5165 |
| Description | Meeting: Making smarter use of household energy data: opportunities and challenges for scaling up research, Dec 2015 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | This meeting was the projects first engagement event held in central London on 9 December 2015. It brought together UK-based energy specialists, statisticians and data scientists from academia, industry, the third sector, and government, including the Department of Energy and Climate Change (DECC) and Office for National Statistics (ONS). The aim was to scope data needs and challenges, and relevant use cases for the project. |
| Year(s) Of Engagement Activity | 2015 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/ |
| Description | Policy Impact interview: Building data infrastructure |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Interview with Policy Impact Magazine, www.impact.pub . Louise Corti from the University of Essex talks about her recently completed project that formed a knowledge exchange and planning network which focused on addressing key policy challenges by sharing the scoping, planning and analysis of UK and South African household energy data. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.ukdataservice.ac.uk/media/605000/ukds-case-studies-delving.pdf |
| Description | Preparing and documenting research data for reuse, Sept 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | This workshop was for researchers at the Agincourt Rural site interested or actively engaged in the creation and management of research data, and looked at the steps required to prepare data for sharing and reuse. It covered existing best practices and tools for data preparation, covering quantitative and qualitative data created in the course of human subject research, including cross-disciplinary data. We track examples of successfully archived (and sometimes complex) datasets, as they make their way through the data assessment, review, processing, curation, and publishing pipeline. It guided participants through data preparation; confidential data management, models of data access, effective documentation practices, and methods of data publishing. |
| Year(s) Of Engagement Activity | 2016,2017 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/11/ |
| Description | Preparing qualitative data for sharing and reuse , August 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | This workshop was for researchers at UCT, Cape Town interested or actively engaged in the creation and management of research data, and looked at the steps required to prepare data for sharing and reuse. It covered existing best practices and tools for data preparation, covering quantitative and qualitative data created in the course of human subject research, including cross-disciplinary data. We track examples of successfully archived (and sometimes complex) datasets, as they make their way through the data assessment, review, processing, curation, and publishing pipeline. It guided participants through data preparation; confidential data management, models of data access, effective documentation practices, and methods of data publishing. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/11/ |
| Description | Smart meter data: quality assurance, June 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | This highly focused workshop invited smart meter researchers to exchange views and methods of quality assurance and cleaning of data. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.ukdataservice.ac.uk/media/604897/datafirstsa_cp_7jul17.pdf |
| Description | Summer School: Encounters with Big Data: An Introduction to using Big Data in the Social Sciences |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The 5 day course, 'Encounters with Big Data: An Introduction to using Big Data in the Social Sciences' was held in Cape Town, South Africa from 30 January to 3 February 2017.The course met one of the objectives of the Smarter Household Energy Data project, a joint International Centre Partnership Grant between the UK Data Service and DataFirst and funded by the Economic and Social Research Council in the UK and the National Research Foundation in South Africa, to create a collaborative research infrastructure for large-scale household energy data. Louise Corti, one of the principal investigators told us: "This course introduced part of our work on 'scaling up' data curation and user access approaches for big data, predominantly where larger or more complex data sources - bigger than the 5GB maximum download bundle of survey data that our users typically access - or where computationally-intensive and iterative modelling is needed." The course covered aspects of extraction, exploration, and statistical analysis of big data behind the UK Data Service's new Data Services as a Platform (DSaaP) initiative being developed in partnership with Hortonworks, a leader in Hadoop-based data technologies. The new platform will be using solutions that deliver data at scale, with speed, and security, including Hive, Spark, and Zeppelin, which integrate seamlessly with popular data analysis environments like R and Python. We received applications for the course from universities across South Africa, including the University of Cape Town, University of Witwatersrand, Stellenbosch University, and the University of Pretoria, as well as from government agencies including the South African National Space Agency and the Human Sciences Research Council of South Africa. The participants were highly experienced in using cross-sectional and longitudinal survey data, public health and medical data, transport data, financial data, and satellite data. Nathan Cunningham, Director of the Big Data Network Support directorate at the UK Data Service, summarised the training: "Our aim was to support researchers in understanding and analysing large and complex datasets, focusing on leveraging the power of popular statistical software like R within a big data environment." |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://blog.ukdataservice.ac.uk/reflections-on-encounters-with-big-data-our-course-in-cape-town/ |
| Description | WITS Rural workshop: Focus on Health and Demographic Surveillance Site data, September 2016 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | In September 2016, 2 members of the project team visited South Africa to run a programme of short workshops. A full day workshop took place at the research site of the WITS Rural Facility (WRF) of the University of Witwatersrand in Bushbuckridge, in the rural north east of South Africa. 12 staff from the local Agincourt Health and Demographic Surveillance site (HDSS) (420 sq km covering 87,000+ people living in 14,000+ households and 26 villages in South Africa's semi-arid rural north-east) and from the Africa Centre for Population Health (also running a similar HDSS, in rural KwaZulu-Natal) attended. Both are part of the broader INDEPTH Network, set up to provide a more complete picture of the health status of communities collected from whole communities over extended time periods, which more accurately reflect health and population problems in low- and middle-income countries (LMICs). Its 43 member centres observe, through 49 HDSS field sites, the life events of over three million people in 20 LMICs in Africa, Asia, and Oceania. These sites are funded through multiple stakeholders, including the Wellcome Trust, NIH, and the EC. The morning session heard from the UKDS and DataFirst speaking about data curation opportunities, especially utilising big data environments, for good data management of more complex data and large-scale analysis. There were questions on the process of getting the Data Seal of Approval and also some clarification provided on the relationship between DataFirst and Stats SA. Louise pointed to the UKDS-ESPA brochure on the Guide to sharing social data in a multidisciplinary, stakeholder research. Other questions arose on: ownership of data; best practices in version control of data and assigning DOIs; which metadata - DDI-Codebook or Lifecycle?; file formats; methods of gaining consent; and models of data licencing and access, such as Secure labs and Safe rooms. Louise drew attention to the UKDS' new 5 Safes animation which explains, succinctly, how the secure access model works, in providing safe access to 'unsafe' data. Following lunch, staff from the HDSS spoke of various challenges and solutions in providing access to the complexity of their data assets from their sites. Mark Collinson, PI on the Agincourt, generously hosted the day. He noted that the event was also a good opportunity for HDSS data groups from different sites to meet and exchange notes on current and future working practices, and think about how to move forward in harmonising some of their data. An example cited by Mark was coding of labour market activity, which is not consistent across the sites (use of ILO classification or not). There are some really positive outcomes from this meeting which all feel represented a first step in better linking up work on data preparation and access being undertaken by HDSS, and the national social science data archives. further, we look forward to working with these HDSS sites to see how big data solutions may facilitate elements of their work. |
| Year(s) Of Engagement Activity | 2016 |
| URL | https://ukdataservicesmartenergydata.wordpress.com/2016/11/10/uk-visit-to-south-africa-research-site... |
| Description | Workshop: Household energy use in Agincourt area, July 2017 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | DataFirst and the UK Data Archive, with the MRC/Wits Agincourt Unit, have been involved in a partnership around making "big data" relevant to social science research in South Africa, in particular in the area of household energy usage. As part of this programme a team has been looking at using the Agincourt HDSS data to think about rural electrification. The purpose of this workshop was to bring together some of the preliminary results of that work and to share it with a broader group, particularly from the Agincourt Unit.; and to consider follow-up research and funding. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.ukdataservice.ac.uk/news-and-events/eventsitem/?id=5119 |