Variational methods in Bayesian inference

Bayesian inference, in its various forms, is one of the core techniques of modern statistics. It is an important method to extract information about complex stochastic systems and their parameters and has successfully been used in many different application areas such as medical trial analysis, bioinformatics, forensic analysis or archealogy. It has also close connections to related fields such as image processing, speech recognition, and neural networks. Research into methods of Bayesian inference is therefore of significant topical interest.The practical implementation of Bayesian inference techniques rests on the ability to calculate high-dimensional integrals. Powerful methods have been developed for this purpose, in particular Monte-Carlo integration schemes, which are stochastic algorithms well suited for numerical computations. Although these methods are flexible and widely applicable, there are many situations where they suffer from significant limitations, e.g., regarding their accuracy and computational efficiency. In recent years, the so-called variational approach has received growing interest as a promising alternative. It is a deterministic approximation scheme that, under certain circumstances, allows to carry out at least some parts of the calculations analytically. It is thus complementary to Monte-Carlo methods and could present advantages in areas where the latter perform badly. Nevertheless, many questions about this approach still need to be explored, for example regarding its precision and reliability, or the development and optimisation of efficient and flexible algorithms.Within this context, this project in its initial stage intends to obtain an overview of the existing variational techniques, their strengths, weaknesses, and areas of applicability, and to compare them to the alternative approaches. In the main stage, the project aims at developing new variational methodology, for example, by applying techniques used in related areas (e.g., machine learning), combining variational and Monte Carlo methods, or improving and extending existing approaches. The results of this work are expected to be relevant not only to the ongoing theoretical research in this field, but also to the practical application and use of Bayesian methods, e.g., as a tool for data analysis.