Statistical physics insights on inference and learning of structured and heavy-tailed datasets

The underlying goal is the study of optimisation and learning in high dimensions using tools from the statistical physics community working on disordered systems. More specifically, the work is about efficiently solving inference problems and gaining insight into static and dynamic properties of learning of structured, non-Gaussian signals using statistical physics methods such as the cavity and replica methods.

The first project is on statistical inference of hidden structures (planted matchings) on hypergraphs using a message-passing and population dynamics algorithms. The latest two completed projects characterse the statisics of the estimators of supervised learning, more specifically generalised linear models for classification (second project) and robust regression (third project) tasks. The method to obtain theoretical predictions of the estimator's statistical properties and performance is a replica calculation. Extensions of the problem considered in these papers are various loss functions, consideration of robust estimators, contaminated Gaussian datasets. They fall into the set of settings that keep the study of the algorithm performance analytically tractable due to the convenient superstatistical (also known as doubly-stochastic, Gaussian scale mixture) construction and thus allow for such generalisations of the popular case of randomly generated Gaussian datasets to non-Gaussian datasets. The next natural step which is currently in progress is the analysis of the dynamical aspects of learning, i.e., characterising the evolution of the estimators during training with non-Gaussian data present.