Enhancing the Data Production of Astronomical Surveys

Lead Research Organisation: University of Southampton
Department Name: Electronics and Computer Science


The increasing data collection and handling capabilities of the astronomer enable the creation of
large scale surveys such as the Sloan Digital Sky Survey (SDSS), Gaia or (in the future) the Large
Synoptic Survey Telescope (LSST). These projects move astronomical datasets in the petabyte
regime, orders of magnitude higher than what has previously been possible. The potential wealth of
information for the astronomer promises large scientific gains meaning that improvements in data
quality and processing efficiency have large potential returns. Future surveys such as LSST, will
likely be particularly important for the study of low mass X-ray binaries (LMXBS). They consist
of a black hole or neutron star accreting matter from a binary companion star. Although there are
thought to be thousands to millions of LMXBs in the Milky Way, only _200 systems have been
observed as the vast majority are obscured by Galactic gas and dust. The high optical sensitivity of
LSST, coupled with the synoptic nature of its observing strategy, makes it the ideal instrument for
discovering more of the LMXB population and characterising their parameters. In this work, the
potential for LSST to recover LMXB periods was investigated with a variety of potential observing
strategies. There was shown to be a factor of _3 improvement in period recovery when using alternative
strategies, rather than the current baseline. Gaia data on LMXBs was also investigated,
here prior information on the distribution on LMXBs was used to increase the accuracy of distance
estimates produced via parallax. Finally it was investigated whether provenance information could
be used to increase the processing efficiency of astronomical data processing. Whilst our results
demonstrated a net decrease in processing efficiency, significant increases were also found when
analysing the final data products.


10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/N509747/1 01/10/2016 30/09/2021
1807985 Studentship EP/N509747/1 29/09/2016 30/09/2019 Michael Johnson
Description I have quantified the performance of several potential observing strategies for the Large Synoptic Survey Telescope (LSST) with respect to Low Mass X-ray Binaries.
I demonstrated the use of provenance as a means of improving the processing efficiency of scientific workflows - contrary to the norm as recording provenance usually adds an initial overhead to the processing. I have identified ~2 million transient events within a set of calibration data for the Kepler Space telescope which include both newly discovered objects and previously known astronomical objects whose observations within this dataset predate their original discovery. I have investigated and quantitatively assessed the use of techniques such as brute force, simulated annealing and the hill climbing algorithm for improving the quality of astronomical data processing. I have developed an approach to be applied to workflows which analyse astronomical data in order to improve the quality of the data that they produce.
Exploitation Route Thee results from period studies and LSST can be extended to other periodic objects and can be used to inform decisions on the final LSST observing strategy when it is operational in ~2022. A tool is under development to facilitate the former. The research into using provenance as a means to improve processing efficiency will help reduce the computational overhead of recording it, making provenance recording more desirable for astronomical workflows. The approach and methods for improving data quality may be implemented by others within the astronomical community.
Sectors Digital/Communication/Information Technologies (including Software),Other

Description UK Involvement in LSST: Phase A
Amount £1,308,664 (GBP)
Funding ID ST/N002512/1 
Organisation Science and Technologies Facilities Council (STFC) 
Sector Public
Country United Kingdom
Start 07/2015 
End 03/2019