Open Soils Data Platform

Lead Research Organisation: Rothamsted Research
Department Name: UNLISTED

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.

Technical Summary

unavailable

Planned Impact

unavailable

Publications

10 25 50
publication icon
Von Fromm S (2020) Continental-scale controls on soil organic carbon across sub-Saharan Africa

 
Title Soil - The Big Data Beneath Your Feet 
Description This the video of the presentation that William Wu made at the GoalKeepers2018 technology event organised by Bill Gates. 
Type Of Art Film/Video/Animation 
Year Produced 2018 
Impact None that are clear. 
URL https://www.youtube.com/watch?v=Fb9R0CnPMkc
 
Description We have developed and implemented an Open Soils data Platform that enables the storage and searching of multiple information sources on soils e.g. soil samples and support materials such as reports for sub-Saharan Africa. The platform uses an agreed set of metadata and controlled vocabularies needed for integrating a range of data. Prototype metadata integration for several data collections has been achieved enabling searching of the catalogue for datasets containing related-information on soils information.The architecture developed is easily extensible to other information resources and databases as soon as they become viable
Exploitation Route Issues emerged regarding access to data from African sources e.g OFRA but discussions with these and others are ongoing. The continuing objectives are to engage with stakeholders to populate the catalogue with additional datasets so it becomes a valued repository of data for soil health for Africa
Sectors Agriculture

Food and Drink

Education

Environment
URL http://catalogue.ceh.ac.uk/osdp/documents
 
Description We have developed and implemented an Open Soils data Platform that enables the storage and searching of multiple information sources on soils e.g. soil samples and support materials such as reports for sub-Saharan Africa. The platform uses an agreed set of metadata and controlled vocabularies needed for integrating a range of data. Prototype metadata integration for several data collections has been achieved enabling searching of the catalogue for datasets containing related-information on soils information. The architecture developed is easily extensible to other information resources and databases as soon as they become viable http://catalogue.ceh.ac.uk/osdp/documents Issues emerged regarding access to data from African sources e.g OFRA but discussions with these and others are ongoing. The continuing objectives are to engage with stakeholders to populate the catalogue with additional datasets so it becomes a valued repository of data for soil health for Africa
First Year Of Impact 2017
Sector Agriculture, Food and Drink,Environment
Impact Types Societal

Economic
 
Title AfSISdb metadata endpoint 
Description This is JSON format endpoint providing the schema and metadata for the AfSIS databases. It is used to populate metadata into the open soils data platform. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact It is part of the infrastructure of the oSDP and is a technical result only. 
URL https://afsisdb.qed.ai/cabinet/api/schema/
 
Title Global Soils metgenomic library 
Description collection of over 450 shotgun metagenome data sets collected from various public repositories and generated from pristine and managed soils around the globe. These are being used to assess the diversity of micro-organisms and functions and study the effects of management upon soil communities.Partnerships with Bioplatforms Australia and the African Soils information service is contributing metagenomes from across Australia and sub-Saharan Africa to this database and extending the utility of the resource. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact none yet 
 
Title Open Data on AWS: Africa Soil Information Service (AfSIS) Soil Chemistry 
Description This dataset contains soil infrared spectral data and paired soil property reference measurements for georeferenced soil samples that were collected through the Africa Soil Information Service (AfSIS) project, which lasted from 2009 through 2018. Georeferenced samples were collected from 19 countries in Sub-Saharan African using a statistically sound sampling scheme, and their soil properties were analyzed using both conventional soil testing methods and spectral methods (infrared diffuse reflectance spectroscopy). 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact The two types of data can be paired to form a training dataset for machine learning, such that certain soil properties can be well-predicted through less expensive spectral techniques. 
URL https://registry.opendata.aws/afsis/
 
Title Open Soils Data Platform 
Description Our aim has been to develop an open data platform for soil information, combining metadata from a broad range of sources and creating a searchable catalogue based upon a semantic data framework. This will enable linking of activities and outputs via common themes (within controlled vocabularies and semantic search) and links across constituent databases using mappings over standard web interfaces and through application programme interfaces (APIs) to the underlying data. This architecture will be readily extensible to other information resources and databases via continual development using common semantics standards such as PROV-O and SoilML. To make the Phase I subset of AfSIS's soil infrared spectral data and paired soil property reference measurements available for release on AWS's Registry of Open Data (RODA), QED (1) coordinated with multiple parties to gather and organize the necessary assets, and (2) built computational notebooks demonstrating how the data can be used. To carry out the first initiative, data from ICRAF and TanSIS, and Phase I wet chemistry data from CROPNUTS, have been uploaded using QED's AfSIS DB portal that has been online since 2014. Re-analyzed AfSIS Phase I wet chemistry data from Rothamsted was sent to QED by way of spreadsheets, which QED then wrote programs for to automatically parse and upload to AfSIS DB. The data in AfSIS DB (https://afsisdb.qed.ai/about/) was then post-processed by way of Python, merging field-acquired georeferences with lab-acquired measurements, and ensuring that only georeferenced samples with both wet and dry chemistry analyses were retained in the final dataset. To collect sufficient metadata, QED also corresponded with ICRAF, CROPNUTS, and Rothamsted to gather formal SOPs for all methods used in their analyses, and these SOPs are also presented on the RODA. In a related effort for propagating metadata, QED also created an endpoint on AfSIS DB that dynamically generates linked data describing the models stored within, to experiment with semantic web technology. The second initiative, of building computational tutorials, has been not only suggested by existing efforts in AfSIS and oSDP, but is also part of AWS's data quality standards for acceptance of a public dataset. Our tutorial first provides an exposition of the structure of the data, and then demonstrates how to read the data from the S3 bucket and build a simple machine learning model for spectral inference. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact We have implemented a first version of the oSDP metadata integration framework based on CEH's Environmental Observatory Framework. This uses an agreed set of metadata and controlled vocabularies needed for integrating a range of data. These have been taken from the CAB Thesaurus and Global Agricultural Concept Scheme and FAO's AGROVOC systems. Prototype metadata integration for several data collections has been achieved enabling searching of the catalogue for datasets containing related information. Some issues have emerged with access to data from African sources (e.g. OFRA) but discussions are ongoing. Metadata descriptions for the chemical and biological datasets (Objective 3) have been developed. Impact is also derived from the importance of releasing data as an exemplar public dataset on the AWS portal. This puts the data in the hands of a major international community who are able to develop new computational methods to interrogate the data. Practical outcomes at end user level in the agriculture and farming community are not available yet, but the international reach of the AWS portal is a major outcome for the project 
URL https://afsisdb.qed.ai/about/
 
Title Open-access data on African soils available via Amazon DB 
Description iGCRF data relating to African soils were used in the first open-access Amazon DB related to soils information. This was announced in December 2018. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact This data will help develop high resolution, ground-truthed maps covering various African regions related to soil chemistry and fertility 
 
Title Wet chemistry data for a subset of AfSIS: Phase I archived soil samples 
Description This dataset contains a subset of the samples collected during the AfSIS Phase I project and was a collaborative effort between World Agroforestry (ICRAF) and Rothamsted Research. The soil samples were retrieved from ICRAF Soil Archive: https://worldagroforestry.org/output/icraf-soil-archive-physical-archive-systematically-collected-soil-samples and subject to wet chemical analysis at Rothamsted Research in the UK under a Global Challenges Research Fund project, "BBS/OS/GC/000014B: Chemical and Biological Assessment of AfSIS soils" funded through the UK Biotechnology and Biological Sciences Research Council. This dataset includes the Site, Cluster, Plot as well as the GPS coordinates and wet chemistry data from 2002 samples collected from 18 countries and 51 LDSF sites. The original data collection was part of the AfSIS Phase I project, funded by the Bill and Melinda Gates Foundation (BMGF) and took place between 2009-2013. ICRAF and CIAT contributed the Site, Cluster, Plot and GPS coordinates for the soil samples, ICRAF organized the sub-sampling of the soil samples from the ICRAF physical archive in Nairobi and Rothamsted analysed the soil samples in the UK in 2017 and 2018. Visit our websites here: https://worldagroforestry.org/landhealth and https://www.rothamsted.ac.uk/. The AfSIS Phase I project funded by the Bill and Melinda Gates Foundation (BMGF) from 2009-2013, aimed to provide a consistent baseline of soil information across sub-Saharan Africa (SSA). Led by CIAT-TSBF, partners included: ISRIC, CIESIN, The Earth Institute at Columbia University and World Agroforestry (ICRAF). ICRAF led the systematic assessments of soil health using the Land Degradation Surveillance Framework (LDSF), which was developed at ICRAF, http://landscapeportal.org/blog/2015/03/25/the-land-degradation-surveillance-framework-ldsf/. LDSF sites were randomized using spatial stratification based on Koeppen-Geiger Climate zones across 19 countries in SSA. In total 60 LDSF sites were sampled. Soil samples were collected using the LDSF at two depths, 0-20 cm (labelled Topsoil) and 20-50 cm (labelled Subsoil). In each LDSF site, approximately 320 standard soil samples were collected. All of these were also scanned using MIR Spectroscopy and are available on Dataverse here: Vågen, Tor-Gunnar;Winowiecki, Leigh Ann;Desta, Luseged;Tondoh, Ebagnerin Jérôme;Weullow, Elvis;Shepherd, Keith;Sila, Andrew, 2020, "Mid-Infrared Spectra (MIRS) from ICRAF Soil and Plant Spectroscopy Laboratory: Africa Soil Information Service (AfSIS) Phase I 2009-2013", https://doi.org/10.34725/DVN/QXCWP1, World Agroforestry - Research Data Repository, V1 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact none yet 
URL https://data.worldagroforestry.org/dataset.xhtml?persistentId=doi:10.34725/DVN/66BFOB
 
Title Fertilizer Optimizer app 
Description The original conceptualization and development of the optimization tool was by the university of Nebraska-Lincoln. Jansen J, Wortmann CS, Stockton MC, Kaizzi CK (2013). Maximizing net returns to financially constrained fertilizer use. Agronomy Journal 105 (3) 573-578. The Fertilizer Optimizer tool was built as part of the Optimising Fertilizer Recommendations in Africa (OFRA) project, led by CABI. The project was a partnership between CABI, the University of Nebraska Lincoln, national governments and agricultural research and extension systems in 13 countries in sub-Saharan Africa. OFRA was supported by the Alliance for a Green Revolution in Africa (AGRA). In this project this spreadsheet-based tool was converted to an Android app and validated against the original spreadsheet version. This included transferring the spreadsheet logic over to an Android app, and providing parameters for crop and the sixty agro-environmental zones. Translation of the interface was completed and as well as English, French and Portuguese are now supported. An upgrade to a mobile phone app now offers farmers across Africa even more benefits and cutting-edge fertilizer use technology. This will help farmers to grow healthier, more productive with increasingly profitable crops, as a result of more informed use of how small amounts of fertilizer impact the crops which they grow. Pilot work on the CABI Fertilizer Optimizer app in Uganda has shown that some farmers realised up to a seven-fold increase in yields. Using funding from the BBSRC Global Challenges Research Fund, the app has now been upgraded to make it easier to use. The new app includes the integrated soil fertility management (ISFM) practices feature, and a calibration tool which helps farmers to apply the right quantity of fertilizer to their crops. The Fertilizer Optimizer app is designed to help resource constrained farmers to maximise the return on investment on fertilizer, based on what the farmer can realistically afford. In the latest version of the Fertilizer Optimizer app it is possible to calibrate the fertilizer recommendation to the planting conditions of a user's field. Version 1 of the app provided users with an amount of fertilizer to use and an application rate (e.g. 5kg per hectare). Now it is possible to enter field measurements, container sizes and preferred application technique and the app will help to evenly distribute the recommended fertilizers across the crop providing actionable information tailored to a particular field. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact The Fertilizer Optimizer has been produced for: Burkina Faso, Ethiopia, Ghana, Kenya, Malawi, Mozambique, Niger, Nigeria, Rwanda, Tanzania, Uganda and Zambia and is available to use in English, French and Portuguese. Impact stories are reported in blog posts attached to this report. The was launched in the Google play store in January and it is too soon to discuss impact. Outreach and marketing campaigns will be undertaken by CABI, the responsible partner for this part of the project. Open Soils Data Platform BBS/OS/GC/200014B We have developed and implemented an Open Soils data Platform that enables the storage and searching of multiple information sources on soils e.g. soil samples and support materials such as reports for sub-Saharan Africa. The platform uses an agreed set of metadata and controlled vocabularies needed for integrating a range of data. Prototype metadata integration for several data collections has been achieved enabling searching of the catalogue for datasets containing related-information on soils information. The architecture developed is easily extensible to other information resources and databases as soon as they become viable http://catalogue.ceh.ac.uk/osdp/documents Issues emerged regarding access to data from African sources e.g OFRA but discussions with these and others are ongoing. The continuing objectives are to engage with stakeholders to populate the catalogue with additional datasets so it becomes a valued repository of data for soil health for Africa 
 
Title Open Soils Data Platform portal 
Description We have developed and implemented an Open Soils data Platform that enables the storage and searching of multiple information sources on soils e.g. soil samples and support materials such as reports for sub-Saharan Africa. 
Type Of Technology Webtool/Application 
Year Produced 2018 
Impact This portal can integrate metadata across data collections and enable searching of the catalogue. 
URL http://catalogue.ceh.ac.uk/osdp/documents
 
Title iPython Notebook - Soil Chemistry Tutorials 
Description An objective of the oSDP project was to build computational tutorials. This tutorial first provides an exposition of the structure of the data, and then demonstrates how to read the data from the Amazon S3 open data portal and build a simple machine learning model for spectral inference. 
Type Of Technology Webtool/Application 
Year Produced 2019 
Impact Too soon for these to be identified. 
URL https://github.com/qedsoftware/afsis-soil-chem-tutorial/blob/master/afsis-soil-chem-tutorial.ipynb
 
Description GoalKeepers 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact William Wu (of QED) was one of two presenters in Bill Gates' technology session during Goalkeepers 2018 conference. This is a live and streamed event highlighting the beneficial impact of technology in advance of the UN General Assembly meeting in New York and a part of the wider Goalkeepers COnference. It was a live presentation about digital soil mapping in Africa and the AfSIS project.
Year(s) Of Engagement Activity 2018
URL https://www.youtube.com/watch?v=Fb9R0CnPMkc