IRIS Digital Asset - UK SRC Proposal to Establish a High-Performance Data Transfer Node Infrastructure at a Supercomputing Site
Lead Research Organisation:
University of Manchester
Department Name: Physics and Astronomy
Abstract
This grant has been awarded to complete a set scope of work for IRIS.
The aim of this proposal for IRIS2021 DRI funding is to build and demonstrate a data transfer infrastructure at the Cambridge IRIS Supercomputer site capable of receiving SKA Observatory Data Products (ODP) from the two telescopes and exchanging Advanced Data Products (ADP) between SKA Regional Centres. This proposal will build on the work done by the Horizon 2020 AENEAS project and the experience gained form ESCAPE, it will also leverages ExCALIBUR investments at Cambridge. Essentially its aim is to combine wLCG File Transfer services with Parallel IO subsystems technologies. The following bullets describe the proposed work:
- Work with the Cambridge IRIS HPC site to configure and install a high-performance Data Transfer Node (DTN). The DTN will be use equipment and connectivity provided for the ExCALIBUR Exascale Data Testbed at Cambridge. This will comprise 100 GE NIC, Memory, NVME disks, Linux (Fedora, Centos, etc.) and associated compilers and development tools.
- Agree the design and implementation of the connectivity for the DTN. This will include the use of a Data Transfer Zone (DTZ) for external network access to the DTN; A DTZ reduces the end-to-end network impediments that affect the speed at which research data can be transferred between experimental, computational and storage facilities. It ensures adequate available bandwidth on the site access link(s) for this work and the consideration of Fire Wall rules for connections between the DTN and site storage.
- Tune the DTN for long-haul high bandwidth transfers; Install memory-to-memory test software and data transfer tools that support http(s) and third-party copy. This will mean installing tools developed for WLCG File Transfer Service (FTS).
- Perform memory-to-memory tests between:
- HPC DTN and GÉANT/JBO DTNs, HPC DTN and other IRIS sites
- HPC DTN over long-haul routes to DTNs in AARnet (Australia), SANReN/Tenet (South Africa) and Canada SRC. Perform low level data transfer tests between the above locations using the HPC DTN disks and the local HPC site parallel IO storage
- Develop and test the software stack to establish the environment at the IRIS HPC site to support the File Transfer System (FTS) and Rucio
- The ExCALIBUR Exascale Data Testbed has a combination of internal highly performant hierarchical storage and highly performant external networking capabilities and is an ideal platform to test end to end data transport to a candidate regional centre and then onto the appropriate part of the filesystem. The ability of the Hierarchical Storage Management (HSM) to move these data quckly into the Supercomputer IO subsystem and to the local Archive will also be measured.
- Test recommendations for integrating DTNs into the IRIS HPC site infrastructure
- The capability of the file system to check data transfer integrity is a key part of this investigation.
The work would be carried out by a collaboration between funded personnel at the IRIS facility at Cambridge and The University of Manchester together with help and advice from GÉANT Association and global NRENs. Members of AARnet (Australia), SANReN/Tenet (South Africa) have agreed to make their DTNs available and allow use to use their networks for the required experimental work. A member of the Canadian Radio Astronomy computer infrastructure has also expressed interest in supporting this work. Members of Manchester, GÉANT Association, and the NRENs have substantial experience of networking, protocols, and long-haul high-bandwidth DTNs.
The requested funding is to support two FTEs each for a 6-month period at £40k per person, that is a total of £80k; they would be located at the IRIS facility at Cambridge and The University of Manchester.
The aim of this proposal for IRIS2021 DRI funding is to build and demonstrate a data transfer infrastructure at the Cambridge IRIS Supercomputer site capable of receiving SKA Observatory Data Products (ODP) from the two telescopes and exchanging Advanced Data Products (ADP) between SKA Regional Centres. This proposal will build on the work done by the Horizon 2020 AENEAS project and the experience gained form ESCAPE, it will also leverages ExCALIBUR investments at Cambridge. Essentially its aim is to combine wLCG File Transfer services with Parallel IO subsystems technologies. The following bullets describe the proposed work:
- Work with the Cambridge IRIS HPC site to configure and install a high-performance Data Transfer Node (DTN). The DTN will be use equipment and connectivity provided for the ExCALIBUR Exascale Data Testbed at Cambridge. This will comprise 100 GE NIC, Memory, NVME disks, Linux (Fedora, Centos, etc.) and associated compilers and development tools.
- Agree the design and implementation of the connectivity for the DTN. This will include the use of a Data Transfer Zone (DTZ) for external network access to the DTN; A DTZ reduces the end-to-end network impediments that affect the speed at which research data can be transferred between experimental, computational and storage facilities. It ensures adequate available bandwidth on the site access link(s) for this work and the consideration of Fire Wall rules for connections between the DTN and site storage.
- Tune the DTN for long-haul high bandwidth transfers; Install memory-to-memory test software and data transfer tools that support http(s) and third-party copy. This will mean installing tools developed for WLCG File Transfer Service (FTS).
- Perform memory-to-memory tests between:
- HPC DTN and GÉANT/JBO DTNs, HPC DTN and other IRIS sites
- HPC DTN over long-haul routes to DTNs in AARnet (Australia), SANReN/Tenet (South Africa) and Canada SRC. Perform low level data transfer tests between the above locations using the HPC DTN disks and the local HPC site parallel IO storage
- Develop and test the software stack to establish the environment at the IRIS HPC site to support the File Transfer System (FTS) and Rucio
- The ExCALIBUR Exascale Data Testbed has a combination of internal highly performant hierarchical storage and highly performant external networking capabilities and is an ideal platform to test end to end data transport to a candidate regional centre and then onto the appropriate part of the filesystem. The ability of the Hierarchical Storage Management (HSM) to move these data quckly into the Supercomputer IO subsystem and to the local Archive will also be measured.
- Test recommendations for integrating DTNs into the IRIS HPC site infrastructure
- The capability of the file system to check data transfer integrity is a key part of this investigation.
The work would be carried out by a collaboration between funded personnel at the IRIS facility at Cambridge and The University of Manchester together with help and advice from GÉANT Association and global NRENs. Members of AARnet (Australia), SANReN/Tenet (South Africa) have agreed to make their DTNs available and allow use to use their networks for the required experimental work. A member of the Canadian Radio Astronomy computer infrastructure has also expressed interest in supporting this work. Members of Manchester, GÉANT Association, and the NRENs have substantial experience of networking, protocols, and long-haul high-bandwidth DTNs.
The requested funding is to support two FTEs each for a 6-month period at £40k per person, that is a total of £80k; they would be located at the IRIS facility at Cambridge and The University of Manchester.
Organisations
People |
ORCID iD |
| Keith Grainge (Principal Investigator) |