Study of regularisation methods in machine learning
Lead Research Organisation:
University College London
Department Name: Computer Science
Abstract
Over the past decade the availability of powerful computers has opened the doors to the use of machine learning techniques in complex application domains such as those arising in computer vision, speech recognition, computational linguistics, marketing science, and bioinformatics, to mention but a few.A central approach in machine learning which has proved valuable in the above domains consists in computing a function from available data by minimising a regularisation error functional which balances different error/penalty criteria. For example, the regularisation error functional may involves the combination of a data term, measuring the empirical error on the data and a penalty term measuring the function complexity. The goal of this visit, during the period of January--June 2006, is to continue to explore both the theoretical and practical implications of the regularisation approach in machine learning as well as produce a first draft of a book on this topic. Prof. Micchelli shares a strong interest with Dr. Pontil in machine learning and the proposed visit will be the first opportunity for them to work together for an extensive period of time.Prof. Micchelli ranks high among the world leaders in computational mathematics. He has made fundamental contributions to that field, especially to problems concerning approximation, representation and estimation of functions. His work has been influential not only in mainstream mathematics but also in nearby fields, particularly in statistics and computer science. He is in the recent ISI list of 200 mathematicians world-wide who are most highly cited.
People |
ORCID iD |
Massimiliano Pontil (Principal Investigator) |
Publications
Micchelli C
(2007)
Feature space perspectives for learning the kernel
in Machine Learning
C Micchelli
(2007)
Representer Theorems for the matrix learning problem.
Argyriou Andreas
(2009)
When Is There a Representer Theorem? Vector Versus Matrix Regularizers
in JOURNAL OF MACHINE LEARNING RESEARCH
Argyriou A.
(2009)
When is there a representer theorem? Vector versus matrix regularizers
in Journal of Machine Learning Research
Argyriou A
(2008)
When is there a representer theorem? Vector versus matrix regularizers
A Argyriou
(2007)
A spectral regularization method for multi-task structure learning
Description | New properties of certain machine learning algorithms based on the minimisation of regularisation functions. Among other results, we derive a necessary and sufficient condition for the validity of the Representer Theorem (RT) in machine learning. The RT is a cornerstone result in machine learning which has been applied widely in the field. Previous work provided only a sufficient condition for the validity of the RT. In this work we prove that this condition is also necessary. This has important consequences on the applicability of the RT to machine learning algorithms. Furthermore, we propose a more general form of the RT which applies to matrix learning problems. Rigour: our analysis is based on advance tools from linear algebra and it requires some non-trivial technical lemmas which are of independent |
Exploitation Route | Several researchers working on machine learning have followed up our work. |
Sectors | Education |
Description | Collaboration with Prof. Charles Micchelli, University at Albany SUNY |
Organisation | University at Buffalo |
Country | United States |
Sector | Academic/University |
PI Contribution | Prof. Micchelli is an expert in approximation theory and convex analysis |
Collaborator Contribution | Prof. Micchelli provided key insight into issues pertaining the study of convex learning algorithm, the study of so-called representer theorem and the analysis of certain regularisers which arise in structured sparsity learning problems and in multitask learning. |
Impact | This is multidisciplinary collaboration across Computer Science and Applied Mathematics Please see the section "Publications" for specific research outputs. |