Scalable, Sample Efficient and Interpretable Bayesian Deep Learning

My work will be focused around the EPSRC's "Artificial intelligence technologies" research area. An abstract can be read bellow.

Neural Networks (NN) are a class of machine learning models which have recently soared in popularity due to their flexibility and scalability to large amounts of data. NNs are most commonly trained using Maximum a Posteriori (MAP) parameter estimation. This framework provides point estimates for model weights, often leading to overfitting, overconfidence in predictions and sample inefficiency. Additionally, NNs often behave like black boxes. Their predictions are notoriously difficult to interpret.

Bayesian methods provide a principled way of tackling the issue of overconfidence in model parameters. In Bayesian Neural Networks (BNN), weight point estimates are substituted by probability distributions. Predictions are made by marginalising the weights, considering all possible parameter values. In these models, uncertainty in weight space is translated into uncertainty in predictions, giving us a way to model 'what we do not know.' Unfortunately, exact inference is often intractable for complex models and approximate inference methods tend to rely on crude approximations which present a trade-off between accuracy of uncertainty estimates and scalability.

My goal is to develop a new class of flexible approximate inference methods for neural networks which are able to fit complex data, produce reliable uncertainty estimates, and scale to large datasets. I want to use the uncertainty estimates produced by these models to automatically generate explanations that are understandable to non-experts about these model's decisions. Finally, I want to use these approximate inference methods to reduce model-bias and build sample-efficient model-based reinforcement learning algorithms.