EXARCHIVE: DPTS Development of Exabyte Scale Data Archiving

EXARCHIVE is a 24 month project to develop a scalable platform and software system for
the long-term preservation, archiving and management of Oil & Gas survey data in
repositories that may range from the hundred Petabyte to Exabyte scale.
Survey and well data are extremely valuable and long-lived: data sets may be required years
or decades after capture. Oil and Gas Exploration and Production (EP) is beginning to move
from physically archiving data tapes offline toward a data chain built on file-based
preservation with access to any dataset on-line, on demand.
EXARCHIVE will develop technology and workflows to keep digital data objects secure, in
multiple copies at different geographic locations, over periods that may be longer than the lifetime
of any particular file format or storage system. It will enable the preservation of not only
the data but also the metadata that defines the archive structure, file formats and processing, to
allow indefinite replication of data sets and their analyses as formats and platforms change
over decades. The approach will conform to the OAIS ISO standard for archival data
preservation. The platform is expected to support highly automated processes for cataloguing,
migration, replication and disaster recovery via remote locations using private Cloud storage
connected by dark fibre networks.
DPTS will offer services, using the resulting system, for data ingest, classification and
cataloguing; data access management; long-term data archiving, storage and recall, with
disaster recovery and duplicated on-line data sets held at remote facilities. There will be
spillover applications to other sectors, including scientific, medical and media archiving.
DPTS, is a well-established specialist provider of data processing, transcription and archiving
for the Oil and Gas EP industry. Experts in distributed computing systems from Imperial
College, London, will provide consultancy and support on advanced research aspects of the
project.