Stereographic Markov Chain Monte Carlo

Motivated by the increasing popularity of Bayesian statistics, Markov Chain Monte Carlo (MCMC) methods have been developed to tackle the problem of approximating posterior probability distributions. By generating a sequence according to a Markov process, we approximate these complicated distributions with the empirical distribution of the sample. Although many MCMC algorithms exist, they often perform poorly against target distributions that are heavy tailed or high dimensional.
To address these issues, Yang, Latuszynski and Roberts (2022) develop MCMC algorithms that utilise the stereographic projection: these methods target distributions supported on Euclidean space by transforming the density onto the d-dimensional sphere. This compactifies the space and allows for greatly improved convergence results. The two existing algorithms are the Stereographic Projection Sampler (SPS), a Random-Walk Metropolis (RWM) based method, and the Stereographic Bouncy Particle Sampler (SBPS), a Piecewise Deterministic Markov Process (PDMP). Both are shown to be uniformly ergodic on a large class of (potentially heavy tailed) target distributions, including multivariate t-distributions with d degrees of freedom. Various heuristics and simulation studies are presented which all suggest that these algorithms behave well even in heavy tailed, high dimensional settings, and can in fact benefit from a "blessing of dimensionality" in some cases. Uniform ergodicity allows the sample paths to make excursions into the tails of distributions before quickly returning to the stationary phase.
The performance of these algorithms is strongly influenced by the choice of the centre and radius/shape parameters of the sphere used in the stereographic projection. To address this, we design an Adaptive SBPS algorithm which automatically updates these parameters as we explore the target distribution, inspired by the AIR MCMC framework from Chimisov, Latuszynski and Roberts (2018). We mimic the proofs in this paper to obtain a WLLN, SLLN and a CLT for the adaptive SBPS. These theorems require the same regularity conditions on the target as the uniform ergodicity results from Yang, Latuszynski and Roberts (2022), as well as an assumption of compactness of the parameter space. In doing so, we introduce a novel method for the study of regenerations in uniformly ergodic, continuous time Markov processes: by dividing the (continuous time) SBPS paths into segments of fixed length, we obtain a "segment chain" in the space of càdlàg functions. This new chain is a discrete time Markov chain, and inherits the uniform ergodicity of the properties from our original process, and can be "split" in such a way to divide the sample paths into a sequence of identically distributed, 1-dependent excursions.
We have now got a working implementation of the Adaptive SBPS algorithm in the Julia programming language. Runs on toy examples suggest that the algorithm performs well even in high dimensional, heavy tailed settings, even when the initial parameters of the algorithm are poorly specified. Many more simulation studies are required, exploring in particular how the algorithm behaves when encountering heterogenous scales in the coordinates of the target, and what advantages/disadvantages our methods have over more traditional algorithms (namely HMC).
Looking forward, there are two possible avenues of research that seem particularly relevant. Firstly, although there is strong heuristic evidence to guide our choice of optimal centre and scale parameters for the SBPS, there are yet to be definitive theoretical results to confirm these choices, even in simple examples. There is also no intuition whatsoever to guide the choice of refreshment rate. It would therefore be of interest to investigate this direction to assist practitioners going forward. Secondly, we could investigate what other traditional (Euclidean) algorithms, such as HMC, could benefit from the stereographic projection.