Models in the Cloud: Generative Software Frameworks to Support the Execution of Environmental Models in the Cloud

Lead Research Organisation: Lancaster University
Department Name: Computing & Communications

Abstract

Cloud computing is one of the most important technological developments of the last few years, with the technology having a major and transformative impact on many areas of society and the economy. For example, cloud computing underpins developments in smart cities, ecommerce and eGovernment and also provides the storage and computational capabilities that underpin data science (cf. big data).
The software support offered by cloud computing is however in its infancy and is tailored towards particular styles of application. This is particularly true with respect to computation in the cloud, where the programming approach offered by the computational framework MapReduce dominates and yet MapReduce assumes a particular style of programming where potentially massive data sets are analysed by a map() operation before results are collated through an associated reduce() operation. This is powerful but very limited.
In parallel, researchers are also interested in realising the benefits of cloud computing in many other areas of application. This project focuses on the support offered by cloud computing to Environmental Science and, in particular, to the execution of potentially complex environmental models in the cloud. This is an area of huge significance, with environmental modelling being the key tool to evaluate uncertainty, risk assessment, and mitigation strategies around flood/ drought, food security and the impact of climate change (with major consequences for the economy and for society). We particularly focus on the principles and techniques in the key areas of Platform as a Service (or PaaS), effectively the middleware for cloud computing. The central insight is that services at the PaaS level need to be more carefully tailored to the needs of key application domains, including but not limited to support for the execution of complex environmental models. We advocate a novel approach based on a combination of model-driven engineering coupled with software frameworks and argue that this enables a paradigm shift in terms of the flexible and tailored support offered by cloud computing for given application domains.
Key beneficiaries of this work include the computer science communities working on model-driven engineering and cloud computing, researchers from environmental modelling in areas ranging from climate change modellers to flood prediction, and also key stakeholders related to environmental management and we include an exciting range of partner institutions from this area to maximise the impact of this work.

Planned Impact

The proposed research is well balanced between cutting edge research and impact on society and the economy. In terms of the latter, we seek a transformational impact on the target scientific community and also on users of environmental models who deal with uncertainty, risk assessment, and mitigation strategies around flood/ drought, food security and climate change. We adopt a multi-faceted impact strategy around the following strands (discussed in more detail below):

1. The establishment of an Experts Group as our key vehicle for end user engagement and dissemination and exploitation (new insights and approaches);
2. Working with appropriate bodies to seek a transformative impact on how Environmental Science more generally is carried out (new paradigms);
3. Working with those involved in big data initiatives in the UK to enhance infrastructure capabilities for environmental modelling (new tools);
4. Working with another research institute (EBTIC) to generalise the results beyond Environmental Science (new avenues).

This is supported by a series of strategic partners, and their their roles are summarised below.

Centre for Ecology and Hydrology (CEH): bringing domain knowledge around environmental modelling and access to NERC cloud infrastructure; supporting strands 1, 2, 3.
Community Earth System Model (CESM) via NCAR: bringing domain knowledge around environmental modelling (climate change); supporting strands 1, 2.
JBA Trust: bringing domain knowledge and application of environmental modelling to a range of real world problem domains; supporting strands 1, 2.
UK Earth System Model (UKESM) via Met Office/ NERC: bringing domain knowledge around environmental modelling (climate change); supporting strands 1, 2.
EBTIC: bringing domain knowledge around next generation networks, MDE expertise, generalisation of results; supporting strand 4.

As well as these impact strands, we also plan public engagement activities which we believe are important in scientific research (will include public lectures and Café Scientifique events).

Publications

10 25 50
 
Description DAFNI (Data & Analytics Facility for National Infrastructure) mid-year review (as member of expert review board) - DAFNI is an EPSRC funded initiative intended to develop a national platform to satisfy the computational needs in support of data analysis; infrastructure research; and strategic thinking for the UK's long term planning and investment needs.
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
URL https://www.dafni.ac.uk/
 
Description IGAC Conference - Paul Young
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact |Paul Young attended the IGAC Conference inTakamatsu, where he talked to a global community of atmospheric chemists. He discussed exploiting the Models in the Cloud concepts with a wide range of people, particularly a growing group of atmospheric scientists based in Africa. The project's developments are particularly relevant for that group, given the comparative lack of high performance computing resources.
 
Description SPARC General Assembly - Paul Young
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Paul Young attended the SPARC General Assembly in Kyoto, where he was talking to a global community of climate scientists and policy experts. Discussions with a global community of atmospheric chemists on exploiting the Models in the Cloud concepts, with an emphasis on how MIC tools could be used in a decision making context.
 
Description UKRI Infrastructure Roadmap Environment Sector Thinking Group meeting
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
URL https://www.ukri.org/files/infrastructure/landscape-analysis-2-pdf/
 
Description WMO Global Atmospheric Watch - Paul Young
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Paul Young attended the WMO Global Atmospheric Watch in Osaka, where he discussed how MIC concepts and outputs could help measurement scientists use sophisticated models to make sense of their data.
 
Description AI 4 Earth grant for Azure
Amount $15,000 (USD)
Organisation Microsoft Research 
Sector Private
Country Global
Start 02/2019 
End 12/2019
 
Description CROP-Net: Monitoring and Predicting the Effects of Climate Change on Crop Yields
Amount £27,753 (GBP)
Funding ID NE/5017208/1 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 02/2019 
End 01/2020
 
Description EPSRC Responsive Mode
Amount £388,701 (GBP)
Funding ID EP/R010889/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 12/2020
 
Description New Approaches to Data Science EPSRC Call
Amount £2,656,400 (GBP)
Funding ID EP/R01860X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 04/2018 
End 03/2023
 
Description CMWR Conference - Opportunities for software... 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Richard Bassett presented 'Opportunities for software engineering practices in deploying environmental models to cloud computing architectures' at the CMWR Conference to a mainly academic audience.
Year(s) Of Engagement Activity 2018
 
Description EGU General Assembly - Models of Models.. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Will Simm presented 'Models of Models!? Using Domain Specific Languages in Environmental Modelling', at the EGU General Assembly to a mainly academic audience.
Year(s) Of Engagement Activity 2018
 
Description EGU General Assembly - Pie in the Sky.. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Study participants or study members
Results and Impact Will Simm presented 'Pie in the sky or models in the cloud? Unlocking cloud architectures for environmental modelling with Model Driven Engineering' at the EGU General Assembly to a mainly academic audience.
Year(s) Of Engagement Activity 2018
 
Description EMS Conference - To InfiniBand.. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Richard Bassett presented 'To InfiniBand, and beyond? Cloud computing for the WRF model, at the EMS Conference to a mainly academic audience.
Year(s) Of Engagement Activity 2018
 
Description Team visit to Rennes to meet at INRIA 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact 2 day visit to Rennes to learn about state of the art Model Driven Engineering tools. As a result of the meeting a workshop has been planned at CMWR in St Malo in June titled 'Application of Advanced Software Engineering Tools and Methods in the Environmental Sciences'. We will be organising this and presenting two papers.
Year(s) Of Engagement Activity 2017
URL http://cmwrconference.org/program/sessions/
 
Description Visit to JRC, Ispra 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Prof. Blair presented the work of the Fellowship and also Models in the Cloud to the EU's Joint Research Centre, in Ispra Italy, disseminating results of this work and also building partnerships for ongoing collaboration. This is important as JRC are at the heart of work to influence European policy based on scientific evidence. The visit took place between 5th and 7th July 2017.
Year(s) Of Engagement Activity 2017
 
Description Visit to thr Informatics Lab, Met Office 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The Ensemble group visited the Met Office, Informatics Lab on 28th November 2018 to discuss common interests around digital technology and environmental change, and to discuss future collaborations, for example around the concept of virtual labs. The Informatics Lab (https://www.informaticslab.co.uk/) has a mission to rapidly develop prototypes which explore how we can use new technology, science and design to make our data useful, and hence there is a significant overlap with the work in Ensemble.
Year(s) Of Engagement Activity 2018
URL https://www.ensembleprojects.org/667-2/
 
Description Workshop on Software Engineering for Sustainability 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A two day workshop was held involving members of Lancaster University and INRIA, held in Rennes on 28th February - 1st March 2017. The purpose was to share our research on the topic of software engineering and sustainability/ living with environmental change, and to look at specific collaboration around the role of model-driven engineering in supporting such work.
This workshop builds on a previous visit by Benoit Combemale to Lancaster University on 5th and 6th December 2016.
It is anticipated that this will be an important ongoing collaboration for the Models in the Cloud project and the greater Living with Environmental Change activities at Lancaster.
Year(s) Of Engagement Activity 2017