(N00014-16-R-FO05) Semantic Information Pursuit for Multimodal Data Analysis

Lead Research Organisation: University of Surrey
Department Name: Vision Speech and Signal Proc CVSSP


In 1948, Shannon published his famous paper "A Mathematical Theory of Communication" [88], which laid the foundations of information theory and led to a revolution in communication technologies. Shannon's fundamental contribution was to provide a precise way by which information could be represented,
quantified and transmitted. Critical to Shannon's ideas was the notion that the content of a message is irrelevant to its transmission, since any signal can be represented in terms of bits.

However, Shannon's theory has some limitations. In 1953, Weaver argued that there are three levels
of communication problems: the technical problem "How accurately can the symbols of
communication be transmitted?", the semantic problem "How precisely do the transmitted symbols
convey the desired meaning?", and the effectiveness problem "How effectively does the received
meaning affect conduct in the desired way?" Hence, a key limitation of Shannon's theory is that it
is limited to the technical problem.

This was also pointed out by Bar-Hillel and Carnap in 1953, who argued that "The Mathematical Theory of Communication, often referred to also as Theory (of Transmission) of Information, as practised nowadays, is not interested in the content of the symbols whose information it measures. The measures, as defined, for instance, by Shannon, have nothing to do with what these symbols symbolise, but only with the frequency of their occurrence." While Bar-Hillel and Carnap argued that "the fundamental concepts of the theory of semantic information can be defined in a straightforward way on the basis of the theory of inductive probability", their work was based primarily on logic rules that were applicable to a very restricted class of
signals (e.g. text). In the last 60 years there has been extraordinary progress in information theory,
signal, image and video processing, statistics, machine learning and optimization, which have led
to dramatic improvements in speech recognition, machine translation, and computer vision technologies.
However, the fundamental question of how to represent, quantify and transmit semantic is what this programme of research shall address.
Description Anomaly detection 
Organisation Bilkent University
Country Turkey 
Sector Academic/University 
PI Contribution Joint research on one class classification methods for applications to anomaly detection
Collaborator Contribution The kernel null-space technique and its regression-based formulation (called one-class kernel spectral regression, a.k.a. OC-KSR) is known to be an effective and computationally attractive one-class classification framework. Despite its outstanding performance, the applicability of kernel null-space method is limited due to its susceptibility to possible training data corruptions and inability to rank training observations according to their conformity with the model. This work addresses these shortcomings by studying the effect of regularising the solution of the null-space kernel Fisher methodology in the context of its regression-based formulation (OC-KSR). In this respect, first, the effect of a Tikhonov regularisation in the Hilbert space is analysed where the one-class learning problem in presence of contaminations in the training set is posed as a sensitivity analysis problem. Next, driven by the success of the sparse representation methodology, the effect of a sparsity regularisation on the solution is studied. For both alternative regularisation schemes, iterative algorithms are proposed which recursively update label confidences and rank training observations based on their fit with the model. Through extensive experiments conducted on different data sets, the proposed methodology is found to enhance robustness against contaminations in the training set as compared with the baseline kernel null-space technique as well as other existing approaches in a one-class classification paradigm while providing the functionality to rank training samples effectively.
Impact Paper submitted for publication
Start Year 2018
Description Biometrics technology demonstration 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact This was an open day organised by the Centre for Vision, Speech and Signal Processing on the occasion of the 30th anniversary of its founding. The research and technologies developed at the Centre were demonstrated to circa 250 attendees, mainly form industry.
Year(s) Of Engagement Activity 2019
URL https://www.surrey.ac.uk/centre-vision-speech-signal-processing/about/30th-anniversary
Description CAIEP-UK Seminar of Foreign Experts 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact This seminar was hosted in London by the State Administration of Foreign Affairs, China to solicit feedback on various aspects of scientific endeavour in China. The feedback may impact on future cooperation in the field of science and technology between China and the UK.
Year(s) Of Engagement Activity 2018
Description Invited keynote talk on Quantifying Semantic Information, presented at the IAPR S+SSPR 2018 Workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Josef Kittler presented an invite keynote talk on Quantifying Semantic Information, at the International Association for Pattern Recognition Workshop on Statistical, Structural and Syntactic Pattern Recognition, held at Fragrance Hill, Beijing, China on August 17-19, 2018.
Year(s) Of Engagement Activity 2018
URL http://ssspr2018.buaa.edu.cn
Description WISE 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Professor Kittler presented an invited talk on aspects of Artificial Intelligence at the Westlake International Symposium 2019. The title of his talk was: The changing face of the curse of dimensionality reduction in machine learning.
Year(s) Of Engagement Activity 2019