MashMyData: Visualization and intercomparison of environmental data on the web

Lead Research Organisation: University of Reading
Department Name: Meteorology

Abstract

There are a number of distinct types of data which environmental scientists use to study the Earth - in-situ data, remotely-sensed data and numerical model output. There needs to be greater integration of these diverse data in order to increase our understanding and minimise errors associated with modelling the Earth system. This can take a number of forms such as model validation, data assimilation, quality controlling of observations and ground-truthing of satellite data. Although these processes are recognised as scientifically desirable, they are often technically difficult, tedious, or time-consuming due to a lack of harmonization across data products - e.g. different data formats, naming conventions, means of data access. We propose to develop a proof-of-concept web portal which will allow the overlaying and comparing of a range of diverse environmental data. Certain standard datasets such as climatologies and databases of marine, atmospheric and terrestrial observations will be accessible automatically. In addition, users will be able to securely upload their own data in a range of formats for comparison with the standard datasets. Users will be able to compare available datasets with each other, for example by calculating the differences between a set of observations and a model. They may then visualise the results of this either on a map, or on a graph of the data at a given location through time. Finally, the user may opt to download datasets in a choice of formats in order to be able to carry out more complex analysis in external applications if desired. This technology will be used in two different areas of scientific research as examples: ocean biogeochemistry and the forecasting of rain.
 
Description This project was to develop prototype software for bringing together ("mashing up") different kinds of environmental data on the web. This involves diverse challenges of visualisation, data interoperability and security.

We developed means of performing data manipulation tasks on physically-distributed data, enabling the scientist to process large datasets without the need to download them or understand complex data formats. The goal of the project was to produce prototype software, which has now been adopted in a number of operational systems.
Exploitation Route Others may adopt the software we developed and may learn from the technical difficulties we faced in making the system work smoothly as a whole. Similar challenges are being faced by numerous other initiatives.
Sectors Digital/Communication/Information Technologies (including Software)

Environment

URL https://code.google.com/p/mashmydata/
 
Title ncWMS - online visualization of environmental data 
Description Provides a means to interactively visualise large multidimensional datasets on the web. 
Type Of Technology Webtool/Application 
Year Produced 2008 
Impact Wide adoption of software by research groups, operational agencies and industry 
URL http://ncwms.sf.net