A fully-parallel alternative to MCMC

This PhD aims to solve complex problems related to applications relevant to the IBM Research laboratory at Daresbury. Specifically, the aim is to develop techniques for implementing state-of-the-art Bayesian techniques in ways that fully exploit the computational power of modern and next generation many-core architectures and systems (such as multicore CPUs, GPUs, Xeon Phis and super-computing clusters).
In Bayesian inference, Markov-Chain Monte Carlo (MCMC) is commonly used for estimating the posterior distribution. With MCMC one can characterise the distribution, and estimate features of posterior distributions that cannot be directly calculated, such as random samples, posterior means, etc. Many researchers have focused on improving MCMC, as they have high computational cost. Previous research involves the use of local gradient information and algorithmic advances. The improved MCMC can effectively solve a majority of problems that can be posed as inferences involving data using statistical models. Nevertheless, one drawback of MCMC is that it cannot exploit parallel processing architectures, limiting its ability to provide solutions to next-generation problems. This happens as MCMC conveys uncertainty by essentially using the evolution of a single Markov Chain. Therefore, MCMC is not ideal for sequential design.
Sequential Monte Carlo (SMC) samplers is an alternative to MCMC that is designed for online inference in dynamic models. Both of these techniques can be used to solve the same problems, with the difference that SMC samplers reduce uncertainty by using the diversity of a set of samples. In SMC samplers, each sample can be processed independently, thus solving MCMC's weakness of not managing parallel processes. Nonetheless, in the process of SMC samplers, it is necessary to perform resampling at a particular time. It is impossible to implement parallel resampling steps in a scalable fashion. In order to do so, in previous studies, researchers have redefined the resampling operation as a divide-and-conquer algorithm. Recent studies indicate that it is possible to leverage the number of cores in order to have faster operation of the resampling algorithm, by taking into consideration data locality and pipelining and by making appropriate use of middleware (e.g., MPI and OpenMP).
The main scope of this research is to develop implementations of an SMC sampler that fully exploit multicore architectures. Specifically, the aim is to use the aforementioned implementations to solve relevant problems, with the hint that these implementations can dramatically outperform MCMC.
This research project is linked closely to a large research project, called "Big Hypotheses", and will pull on previous work related to high-performance computing, Big Data and Bayesian statistics.