Towards improved, safe and robust experimental decision making augmented by generative modelling and synthetic environments

Adaptive experimental design is a critical aspect of many fields including science, engineering and policy making (Ivanova & Hedman et al 2024). Methodologies that support non-myopic decision making- decisions that look ahead to future outcomes rather optimising for the next step greedily- are essential for the effective acquisition of informative data under resource constraints (e.g. time and money).

This research will explore the paradigm of optimal non-myopic decision making, focusing on ensuring robust, safe action selection. Much of this work leverages tooling across Bayesian experimental design (BED), information theory, reinforcement learning (RL) and recent advances in generative architectures, seeking to develop improved methodologies for adaptive experimentation.

The initial focus is on leveraging BED, a structured approach to the optimal experimental design question (Foster, 2021). BED provides a statistical framework to model the problem at hand and parameterises the data generation process with quantities of interest about which we would like to learn. Learning is characterised by the mutual information between observations and underlying parameters, often alternatively framed in terms of expected information gained (EIG). By adopting a BED-based approach, the growing body of BED literature on amortised decision making (Foster et al, 2021) can be built upon. Within this research, initial contributions to literature will focus on trading the benefits of traditional greedy BED with the non-myopic benefits of fully amortised BED typically unlocking faster implementation. Additionally, methods to ensure privacy and safety over sensitive parameters will be explored which may require further research into information theoretic measurement approaches.

Existing methods for estimating mutual information primarily focus on lower bound estimation (Poole et al, 2019). They vary in complexity from simpler, poor performing bounds which rely on prior samples (e.g. PCE) to more sophisticated approaches like ACE which attempt to learn a variational posterior. Specifically, in this work it may be necessary to develop bounds on expected information gained under alternative models and also approaches to minimize EIG - requiring upper bounds.

Following this initial foundation, the work has the potential to go in two potential directions:

1)Real-World Applications of BED leveraging Generative Models: Searching for real world applications for these methodologies, perhaps leveraging increasingly powerful generative models (e.g. diffusion models) as real world simulators or for data augmentation. These models have already shown promising results in fields like medical imaging, and applying them to real-world experimental designs could enable optimized decision-making in critical areas such as treatment planning. Synthetic approaches become most critical when real-world data is limited or expensive to acquire.

2) Bridging BED and RL: There are growing parallels between BED and RL and opportunities arise to bridge the literature. Specific examples include framing BED within a Markovian context (Blau et al, 2022) which unlocks the vast engineering-optimised decision-making policies from RL. Additionally, information theoretic reward objectives -common in BED- could be cast within RL particularly for decision under uncertainty