Adaptive Stereographic Markov Chain Monte Carlo

Motivated by the increasing popularity of Bayesian statistics, Markov Chain Monte Carlo (MCMC) methods have been developed to tackle the problem of approximating posterior probability distributions. By generating a sequence according to a Markov process, we approximate these complicated distributions with the empirical distribution of the sample. Although many MCMC algorithms exist, they often perform poorly against target distributions that are heavy tailed or high dimensional.

To address these issues, Yang, Latuszynski and Roberts (2022) develop MCMC algorithms that utilise the stereographic projection: these methods target distributions supported on Euclidean space by transforming the density onto the d-dimensional sphere. This compactifies the space and allows for greatly improved convergence results. The two existing algorithms are the Stereographic Random Walk (SRW), a Random-Walk Metropolis (RWM) based method, and the Stereographic Bouncy Particle Sampler (SBPS), a Piecewise Deterministic Markov Process (PDMP). We have since introduced the Stereographic Slice Sampler (SSS), based on the geodesic slice sampler from Habeck et al (2023). All three algorithms are shown to be uniformly ergodic on a large class of (potentially heavy tailed) target distributions, including multivariate t-distributions with d degrees of freedom.

Various heuristics and simulation studies are presented which all suggest that these algorithms behave well even in heavy tailed, high dimensional settings, and can in fact benefit from a "blessing of dimensionality" in some cases. Uniform ergodicity allows the sample paths to make excursions into the tails of distributions before quickly returning to the stationary phase. However, The performance of these algorithms is strongly influenced by the choice of the centre and radius/shape parameters of the sphere used in the stereographic projection.
To address this, we design an adaptive versions of the stereographic MCMC algorithms which automatically updates their parameters as we explore the target distribution, inspired by the AIR MCMC framework from Chimisov, Latuszynski and Roberts (2018). We mimic the proofs in this paper to obtain L2 and almost-sure convergence results, and a CLT for all three algorithms. These theorems require the same regularity conditions on the target as the uniform ergodicity results from Yang, Latuszynski and Roberts (2022), as well as an assumption of compactness of the parameter space.

Due to a lack of study of adaptive versions of continuous time MCMC algorithms, we introduce a novel method for the study of regenerations in uniformly ergodic, continuous time Markov processes: by dividing the (continuous time) SBPS paths into segments of fixed length, we obtain a "segment chain" in the space of càdlàg functions. This new chain is a discrete time Markov chain, and inherits the uniform ergodicity of the properties from our original process, and can be "split" in such a way to divide the sample paths into a sequence of identically distributed, 1-dependent excursions. Any uniformly ergodic Markov process (discrete or continuous) can be placed into the segment chain framework, and therefore automatically inherits all of our convergence results.

We have performed several simulation studies comparing the three algorithms' performance against each other and Euclidean HMC. We find the SSS to perform best, since the SRW is algorithmically slow and the SBPS is computationally expensive.

Going forward, we currently have two main aims. Firstly, we wish to find a weakness in the SSS, in order to better understand its overall performance. Secondly, we wish to investigate the impact that the choice of refreshement rate has on the SBPS algorithm's mixing properties, as this has not been studied so far. Beyond this, we are also interested in constructing new stereographic MCMC algorithms, such as potentially a stereographic HMC.