Newton STFC-NARIT; Data Flow and Archiving for Robotic Operations Present and Future

Data archiving is axiomatically fundamental to the success of the knowledge economy. Where a primary production economy depends primarily on physical labour, there can be no doubt that the ability to efficiently manipulate and represent data to scientists, administrators and lay people is critical to any economy built on the value of human knowledge, analysis and experience.

Our own discussions with Thai researchers explicitly identified dissemination and coordination of information as a major stumbling block to effective and efficient governance and development in their economy. The Thai government collects much useful data but there is insufficient breadth of experience in the available workforce on matters such as database integration and exploitation in order to extract value from those data.

Looking further ahead, NARIT and LJMU share the common goal of collaboratively developing the world's largest and fastest responding robotic telescope which will address a critical capability gap foreseen in the global astronomy community for the 2020-2040 timescale. This new telescope focuses on exploitation of the anticipated new era in time domain astrophysics. For that to succeed, for either partner, UK or Thailand, dynamic and efficient data distribution is absolutely critical rather than simply desirable. Data management and distribution is a key work package for the LT2 project, so the Newton funded project proposed here will enable NARIT to be a developmental partner in this major piece of research infrastructure. Data handling is absolutely fundamental to any modern knowledge economy, so this proposal will develop widely transferable skills for both partners.

In this context, astronomy is purely one specific use case upon which to base training and capacity building in data flow management and archiving. It provides a convenient model and test bed for commercial and governmental data handling because there are no concerns about accidental disclosure of personal or confidential information. It is however global in scale involving collaboration of partners around the World and requires handling of diverse and high precision data sets. For example, storage is required for heterogeneous data sites such as imaging and spectroscopy and these need referencing with derived data products such as photometric catalogues.

The scope of the proposed project incorporates a survey of other observatories' and data servers' capabilities and software deployments, but more importantly, looks beyond software technologies already in use with astronomical databases. The objective is to assess and optimise the use of modern standards and technologies, such as relational vs non-relational databases (MySQL, Postgres vs. NoSql etc.) and multi-tiered software stacks with discrete database management, middleware and user interface layers. Astrophysics is just one implementation example of these highly transferable skills and technologies which have far reaching application globally in any knowledge-based economy.

Being the core objective of the grant, industrial and economic impact within Thailand is discussed in the "Case for Support". Whilst some of the same arguments could be made, the UK economy is in a very different position on the world stage and thus the targeted impacts are different. The new archive's major societal significance is to the National Schools' Observatory and other public outreach activities. The NSO currently has about 3000 registered schools (primary, secondary and FE), the website attracts more than 1,500,000 visits per year and we have dealt with over 16,000 observing requests in the last year. Our existing science archive is used by the NSO to serve LT data to schools across the UK but the scope and scale of the NSO has grown enormously in the past decade, changing many of the originally foreseen requirements, and we will work closely with them to ensure optimum schools' use is integrated into the software design. To deliver our strategy, collaboration between the engagement and research teams is essential, with mutual benefits.

A related benefit of developing a more streamlined and open archive infrastructure is increasing the global visibility of UK astronomical research both within and outside the academic community. Increased integration with the virtual observatory advertises the availability of UK data and expertise to professional astronomers. Through the Liverpool Telescope, among other observing facilities, the UK has a leadership in robotic observatory operations and time domain astrophysics and increased public availability of these data advertises UK leadership in this highly technical and inspirational field of science.