UK Gaia CU9: Delivering Gaia to the Community: 2019-2024

Gaia is the cornerstone ESA mission which provides the first astrometric census of the sky. More than two billion sources will be surveyed; allowing major advances in Galactic structure and evolution, stellar evolution, solar system dynamics, exo- planetary systems, cosmology and fundamental physics. Gaia is launched in autumn 2013, with the Gaia Data Release 2 in Apr 2018, representing the first major release of full astrometry for over 1.3 billion sources.. Its huge data set will revolutionise much of astrophysics, provided astronomers can access and understand the data. This proposal is to support the UK leadership roles in Gaia data delivery to users, especially in archive design, science requirements specification, documentation, and support for public-access interface software.

While the primary motivation for Gaia is to revolutionise our understanding of our Universe, Gaia is also a software and technology mission. Gaia UK technology development has two aspects. Industrial developments, for example of new classes of CCDs, ESA-funded at e2v, are already proving of wider importance, have earned the UK the Euclid contract, and are generating other new international markets. Precision control systems, and on-board real-time image processing systems (Astrium Stevenage) are intended to be applied much more widely. There are many ESA-UK industry-UK academic partnerships built on Gaia. The first Gaia data releases have underpinned significant outreach activities. The Gaia data are underpinning a range of deep learning investigations, and are being used in PhD training programmes, for instance the STFC funded CDT network in Cambridge.

The majority of development however involves fast complex processing of huge data volumes, and the development of tools to allow fast public access to huge and complex data sets. This naturally builds on and expands the Big Data and virtual observatory developments of late. The Cambridge developments in very large database systems (Spark/ Hadoop) have been adopted mission-wide, illustrating one more area of technical leadership. Gaia is the next `big beast` in data volumes, with Gaia learning experience showing applications to astronomy (Euclid/ PLATO) and wider implications for science and aspects of the digital economy. Big Data problems generically involve gathering subsets of the data for further exploration and visualising those data in some fashion. The techniques already in use in TOPCAT provide some important software approaches, and the developments to be driven by this work will be applicable to other fields, for example visualising remote sensing data monitoring crop rotation, relevant to areas such as the STFC Food network.