Geometrical Methods for Statistical Inference and Decision
Lead Research Organisation:
University of Cambridge
Department Name: Pure Maths and Mathematical Statistics
Abstract
The important problems of statistics concern what we can learn from empirical data, what might happen next, and what is the best course of action. Statistical inference is the process of extracting information about the underlying nature of the data, to allow us to make predictions about future events. Statistical decision theory searches for strategies that will lead to optimal outcomes, taking into account the intrinsic uncertainty in our predictions. For instance, data on the response of patients to a pharmaceutical drug enable us to infer the effectiveness of the drug in the general population. Given a desirable social goal, such as maximising health benefits while minimising adverse reactions, we may then decide what treatment allocation and dosage levels would be optimal.It is remarkable fact that such statistical questions can be reframed in the mathematical language of geometry. To be more precise, geometric descriptions of objects, involving e.g. distances between points, can be applied to statistical models. However, instead of thinking of an object as a collection of points in 3-dimensional space, the points are now the various different probability distributions that could generate the data. To take a simple example, optimal estimation becomes the process of finding the geometric point which represents the best fitting distribution, and of quantifying how close it is to the true distribution generating the data. More sophisticated applications utilise e.g. the geometric curvature of the statistical model to quantify the uncertainty in our inferential conclusions.The geometric approach to statistical inference has been intensively studied, but there has been little attempt to apply it to statistical decision theory. Building on theoretical foundations recently laid down by Dawid and Lauritzen, this project will develop new theory and applications of geometric decision analysis. In particular it will introduce geometric concepts and techniques originating in Physics and Cosmology to the study of problems of statistical inference.
Organisations
| Description | The concept of a scoring rule is applicable to both statistical inference and statistical decsision theory. For this reason, scoring rules were a natural starting point for our work. The results we obtained follow from one simple organising principle: that a scoring rule should depend only on the observed data and data that is, in a sense to be made precise, near to the observed data. We call such scoring rules "local". A remarkable fact, which is true for all local scoring rules, is that the quoted probability distribution need not be normalized. This is remarkable because the normalization is vital in many statistical applications, e.g. maximum likelihood estimation, and is the main quantity of interest in statistical physics. For continous outcome spaces, we were able to characterize essentially all local scoring rules. Such scoring rules depend on the probability density and its derivatives. We found a connection to the time-independent Schroedinger equation and, as a consequence, a fascinating relationship to a recent approach to clustering in data mining. Another nice feature of these scoring rules is that they are invariant to invertible transformations of the outcome space. Scoring rules can also be made robust. Locality on discrete outcome spaces is rather more flexible: nearness is defined by an arbitrary undirected graph on the outcomes. The specification of local scoring rules in this case then follows from a lovely connection to the factorization theorem for the joint probability distribution of random variables on a graph. Because the graph on the outcomes can be specified by the user, applications to missing data problems and sequential prediction are possible. Furthermore, the well known pseudolikelihood approach turns out to be an example of a local scoring rule. Local scoring rules have a natural geometrical structure. We found the metric determines the Godambe efficiency of estimators derived from the scoring rule. Furthermore, the metric appears, in some cases, to share with the Fisher metric the property of being invariant under sufficient reduction of the data. The role of curvature is the subject of continuing investigation. |
| Exploitation Route | Local scoring rules make statistical estimation possible in a number of statistical areas and hence are ripe for application. Their connection to information and decision geometries are currently being explored. |
| Sectors | Agriculture, Food and Drink |
| Description | Our findings have been used in the statistical and machine learning communities. From a theoretical perspective, they have been used to unify and justify a number of existing statistical techniques. From an applied perspective, they have been used in plant epidemiology and in time series analysis. |
| Sector | Agriculture, Food and Drink |