Semiparametric Bayesian Estimation in Latent Variable Models

The current work for my DPhil involves the development of novel theory for Hidden
Markov Models (HMMs). This work is undertaken collaboratively between my supervisor, Judith Rousseau, and myself. The applications of such models are wide-ranging, including signal processing, natural language processing, economics, molecular dynamics and
biochemistry.

In modern statistical applications, Bayesian nonparametric approaches are ubiquitous
for their flexibility, their key element being the posterior distribution which quantifies our beliefs about the parameters of the model once the data has been observed, and which allows for point estimation and uncertainty quantification. One of the fundamental
elements of this method of inference is the prior distribution, which alongside the data informs this posterior distribution. The prior distribution itself contributes to the flexibility of this family of methods, by allowing us to construct models in a hierarchical
way, and by allowing us to favour more straightforward explanations of the data generating process by penalising overly complicated parameters.

However, the choice of prior in Bayesian methods is a delicate task in these nonparametric
models. It is already known that inference can be overly dependent on this choice, which becomes less and less interpretable as the model grows - thus making the classical interpretation of the prior as an `expert prior opinion' less and less valid. This marks
a clear need for theoretical results in this area, which will inform the choice of priors when the parameter space is difficult to understand, to ensure that the application of the resulting posterior distribution as an inferential tool is placed upon a firm
mathematical foundation.

We are developing theory which will provide guarantees for estimation procedures in
such models - specifically, we are currently working towards a `Bernstein von-Mises' result for HMMs which will inform prior choice to provide theoretical soundness to the use of the posterior for both of the aforementioned ends. In the development of this
theory, we will also show the existence of estimation procedures for this model, which will themselves also be shown to be theoretically valid. The results which we will work towards are termed `semiparametric', as they involve the estimation of a smaller
parameter within a larger (nonparametric) model, and are the first of their kind in the HMMs context.

In the future, we will begin work in collaboration with another of J. Rousseau's students,
on the development of novel theory for Hawkes processes. These `self-exciting' processes have wide-ranging applications from seismology, to finance, to neuroscience and even to epidemiology.

It is only in recent years that their theoretical properties have begun to be understood,
but there still remains a wide range of interesting and important theoretical questions which one can ask in relation to these models. As with our work for HMMs, the goal of this research will be to develop Bernstein von-Mises results which provide theoretical
guarantees for the use of the posterior distribution as a tool for estimation and uncertainty quantification.

This research falls under the EPSRC theme of Mathematical Sciences, in the category
of Statistics and Applied Probability.

The primary CDT impact will be training 75 PhD graduates as the next generation of leaders in statistics and statistical machine learning. These graduates will lead in industry, government, health care, and academic research. They will bridge the gap between academia and industry, resulting in significant knowledge transfer to both established and start-up companies. Because this cohort will also learn to mentor other researchers, the CDT will ultimately address a UK-wide skills gap. The students will also be crucial in keeping the UK at the forefront of methodological research in statistics and machine learning.
After graduating, students will act as multipliers, educating others in advanced methodology throughout their career. There are a range of further impacts:
- The CDT has a large number of high calibre external partners in government, health care, industry and science. These partnerships will catalyse immediate knowledge transfer, bringing cutting edge methodology to a large number of areas. Knowledge transfer will also be achieved through internships/placements of our students with users of statistics and machine learning.
- Our Women in Mathematics and Statistics summer programme is aimed at students who could go on to apply for a PhD. This programme will inspire the next generation of statisticians and also provide excellent leadership training for the CDT students.
- The students will develop new methodology and theory in the domains of statistics and statistical machine learning. It will be relevant research, addressing the key questions behind real world problems. The research will be published in the best possible statistics journals and machine learning conferences and will be made available online. To maximize reproducibility and replicability, source code and replication files will be made available as open source software or, when relevant to an industrial collaboration, held as a patent or software copyright.