Semantic Information Pursuit for Multimodal Data Analysis
Lead Research Organisation:
University College London
Department Name: Computer Science
Abstract
Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
Publications
Bulathwela S
(2020)
TrueLearn: A Family of Bayesian Algorithms to Match Lifelong Learners to Open Educational Resources
in Proceedings of the AAAI Conference on Artificial Intelligence
Bulathwela S
(2023)
Semantic AI in Knowledge Graphs
Bulathwela S
(2022)
Power to the Learner: Towards Human-Intuitive and Integrative Recommendations with Open Educational Resources
in Sustainability
Bulathwela S
(2020)
Towards an Integrative Educational Recommender for Lifelong Learners (Student Abstract)
in Proceedings of the AAAI Conference on Artificial Intelligence
Bulathwela S
(2020)
SUM'20: State-based User Modelling
Bulathwela S.
(2020)
Predicting Engagement in Video Lectures
in Proceedings of the 13th International Conference on Educational Data Mining, EDM 2020
Chérief-Abdellatif B
(2022)
On PAC-Bayesian reconstruction guarantees for VAEs
Grant E
(2018)
Hierarchical quantum classifiers
in npj Quantum Information
Guedj B
(2020)
Kernel-Based Ensemble Learning in Python
in Information
| Description | The work on the generalisation analysis has followed a number of approaches towards gaining insights into the performance of learning systems studied within the proposal. The main developments have been made within the PAC-Bayesian framework for the analysis of generalisation (see Guedj, 2019, for a primer on PAC-Bayesian learning that formed the backbone of the ICML 2019 tutorial given by Guedj and Shawe-Taylor) and have addressed the current open problems in the study of the semantic information pursuit of multimodal data analysis. These concern issues such as the unexpectedly good performance of over parametrised systems such as deep networks, unsupervised learning of manifolds and clustering, invariance in learning systems, ensemble methods for multimodal data, in addition to how such analyses can inform algorithms to optimise performance. The first papers concern the analysis of deep learning and their exploitation. Rivasplata et. al. (2018) involves the extension of PAC-Bayes analysis to include data-distribution defined priors for the case of Support Vector Machines. This work has linked PAC-Bayes analysis with the stability approach to bounding generalisation, significantly tightening the bounds obtained when compared to the original stability bounds derived by Bousquet and Elisseeff. More importantly the work links with studies that have suggested a tight connection between stability and the generalisation of stochastic gradient descent (SGD) trained deep networks. This is closely related to the analysis proposed by Achille and Soatto at UCLA for analysing the information encoded in deep learners. Singh and Shawe-Taylor (2018) also considered the generalisation of deep learning but used the independently and identically distributed (i.i.d.) property of the mini-batch to choose a direction for which we can generate generalisation bounds on the effect of the update on the whole training and test sets. We are thus applying generalisation analysis to the effect of a particular mini-batch update. By ensuring these bounds remain tight we are able to provide bounds on the generalisation of the trained network after a relatively small number of iterations. Hence, this work has the potential to deliver tight bounds for a novel deep learning algorithm that promises to be very data efficient based on initial experiments. Letarte et al. (2019) presents comprehensive study of multilayer neural networks with binary activation, relying on the PAC-Bayesian theory. It develops an end-to-end framework to train a binary activated deep neural network and provides non-vacuous PAC-Bayesian generalization bounds for binary activated deep neural networks. The analysis inherently overcomes the fact that binary activation function is non-differentiable and the performance of our approach is assessed on a thorough numerical experiment protocol on real-life datasets. We have explored the use of PAC-Bayes bounds to train probabilistic neural networks (Rivasplata et al., 2020), aimed at quantifying prediction uncertainty. We have shown great improvements with respect to previous work and how they can be used with promising results in deeper architectures. We intend to test such probabilistic networks in combination with the architectures considered in visual scene graph generation, that produce a graph explaining a visual scene. We believe accounting for uncertainty in such scenarios will improve the performance of these methods. Other work developing the theory of PAC-Bayesian learning is included in Mhammedi et al. (2019), Guedj and Pujol (2019) that develop new bounds as well as the meta analysis of the importance of prior assumptions and modelling. The second group of papers is concerned with aspects of unsupervised learning including manifold learning and clustering. Chretien and Guedj (2019) looks at matrix factorisation and develops a reconstruction bound with a Langevin Monte Carlo algorithm that can efficiently implement the PAC-Bayes inspired approach. Cohen-Addad et al. (2019) analyses online clustering in high dimensional stream of data, a condition of importance to the potential application domains of the project. Finally, Nozawa et al. (2019) considers contrastive unsupervised learning (CURL) for inferring a representation from unlabelled data. The paper extends the framework to flexible PAC-Bayesian setting establishing generalisation bounds for CURL that drive a new representation learning algorithm that shows competitive accuracy while yielding non-vacuous bounds. The final group of papers concerns ensemble methods and analysis of multi-modal data. Guedj and Desikan (2020) provide a new supervised learning approach for aggregating learners while Guedj and Rengot (2020) consider such a non-linear aggregation of filters to improve image denoising. The approach is supported by a theoretical analysis and experiments that demonstrate the efficacy of the approach. Finally, Sun et al. (submitted) provide an analysis of multi-modal learning using PAC-Bayesian analysis based on a data distribution defined prior. |
| Exploitation Route | There are a range of further work particularly focussing on the PAC-Bayes analysis of complex learning phenomena. |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Description | Graph Parameters and Physical Correlations: from Shannon to Connes, via Lovász and Tsirelson |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | Article in ERCIM news |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://ercim-news.ercim.eu/en112/special/graph-parameters-and-physical-correlations-from-shannon-to... |
| Description | ICML Tutorial |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Benjamin Guedj and I gave a tutorial on A Primer on PAC-Bayesian Learning at ICML 2019 one of the two premier conferences in machine learning and artificial intelligence. It was attended by approximately 750 people and was streamed live to a wider audience, see https://icml.cc/Conferences/2019/ScheduleMultitrack?event=4338. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://icml.cc/Conferences/2019/ScheduleMultitrack?event=4338 |
| Description | NeurIPS Tutorial |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | It was an invited tutorial at the premier machine learning conference, Neural Information Processing Systems held in December 2018 in Montreal. The audience was over 500 researchers and professional practitioners from industry and business. |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://www.youtube.com/watch?v=m8PLzDmW-TY |