Statistical machine learning for efficient hidden parameter estimation and decision making

Real-world problems are often considered black-box systems, as we can observe the output but have little information about the inputs and inner computations. Since we can not directly "open the box", we resort to estimations of these values. However, the probabilistic nature of these systems limits the ability of deterministic estimation algorithms. Thus, we take a Bayesian approach and estimate the probability density of the output. We can further attempt to parameterise this density by estimating the latent variables that produce the density. These can be considered the input of our black box.

The sample data in most of these problems are incredibly high-dimensional. Unfortunately, density estimation of high-dimensional data results in a tradeoff between accuracy and efficiency. This problem is also known as the "curse of dimensionality". Leveraging the power of deep learning to estimate density is a novel area of research. The vast explanatory power of neural networks combined with efficient implementation through modern hardware shows promise to accurately and efficiently estimate these densities.

Our research aims to improve the accuracy and efficiency of density and latent variable estimation through exploring the state of the art Bayesian machine learning techniques. More specifically through year one, we will explore integrating Variational autoencoders with neural density estimators for variational inference and latent variable estimation. Through years two and three we currently plan to compare methods similar to neural density estimation, such as auto-regressive flows. Furthermore, we will explore a wide range of applications, from geoscience to biology, to fully test our techniques.

We expect the project to improve the efficiency and reliability of parameter estimation and the decision-making process for complex industrial applications, particularly high dimensional. Real-world problems with known input and internal computations are a small subset compared to most that have the aforementioned black box systems. As such, this allows the applications of our research to be far-reaching.

The relevance of our research to EPSRC is through the development of novel computational methods that will provide solutions across all fields handling complex data.