Semi-Automated Checking of Research Outputs

Trusted Research Environments (TREs) play a vital role in enabling researchers to analyse confidential data such as health records then report findings. The Five-Safes
Framework is used to ensure data confidentiality and includes Safe Outputs. Outputs are typically checked by two expert staff before release, which is a significant expense for TRE
operators, and can cause a bottleneck for researchers.
Meanwhile, the parallel development of TREs and understanding of disclosure-risk, has
created a need to consolidate theory and practice to minimise inconsistent behaviour
between TREs. Addressing both these issues, this project seeks to reduce the operating costs of TREs, and the time taken to release research results. It will:
•Produce a consolidated framework with a rigorous statistical basis that provides
guidance for TREs to agree consistent, standard processes to assist in Quality
Assurance.
•Design and implement a semi-automated system for checks on common research outputs, with increasing levels of support for other types such as AI.
•Work with a range of different types of TRE in different sectors (health, social data)
and organisations (academia, government, private sector) to ensure wide applicability.
•Work with public and patients to explore what is needed for public trust that any
automation is acting as “an extra pair of eyes”: supporting not supplanting TRE staff,
helping them to make easy decisions more rapidly and therefore focus on more complex or nuanced cases.

Trusted Research Environments (TREs) cannot function without appropriate output
statistical disclosure control, and DARE has rightly recognised that “There are already
significant issues with staffing resources to support statistical disclosure control (safe outputs). This staffing issue is acting as a barrier to scaling up the use of TREs for
research” [3, p59]. This proposal builds on existing work in this area, the vast majority by
the proposers [1, 2, 6, 7, 9 - 14], to investigate and advance a framework for automating components of TREs’ disclosure control process to support a consistent, efficient, and trustworthy approach that lets them “focus … on the areas of most significant risk” [3].

To achieve consistency, within and between TREs, we will consider processes as well as
algorithms - recognising that TREs differ in risk appetites, but these need to be expressed in a consistent fashion.
• WP1 will bring together the latest theoretical advances to provide a coherent intellectual framework and resources including practical guidelines for disclosure control in different technical and procedural environments

To achieve efficiency and transparency we will create resources that help researchers
make fewer disclosive requests and TREs spot these faster. The design framework and tools will:
• Support researchers to use the major analytical languages (R, Python and Stata), with minimal changes, by exploiting the ‘wrapper’ approach we have successfully
trialled elsewhere [7, 10] (WP2)
• Support TREs with different operating models and output checking workflows,
through a process of co-design to maximise useability [6] (WP3)
• Automate checking of most common statistics, using best-practice principles-based modelling [15],
• Build on the GRAIMatter project [9] to create practical guidelines and prototype tools
for triaging AI models for clearance (WP5)
To achieve trustworthiness, WP4 builds on our existing PPIE arrangements to explore public attitudes to how disclosure control is currently managed and how conceptual and technical developments, especially around automated decision making, need to be framed to ensure and maintain trust.

To achieve impact, WP6 will build on our extensive UK and global networks of TRE practitioners, to conduct a series of workshops and ‘hands-on’ evaluations to ensure the design frameworks support buy-in from a wide range of prospective users across health and social sciences, and from the public and private sectors.