FISH: Fast Semantic Nearest Neighbour Search

This proposal addresses Theme 1 of the Data Intensive Systems (DaISy) Call "Extracting meaningful information: Deriving meaning from large, heterogeneous, incomplete, contradictory, noisy and dispersed data sets with many different forms and formats (e.g. text, image and sound files)". The project will research and develop novel fast semantic nearest neighbour search (FISH) data structure, algorithm and software utilities for rapidly discovering semantically similar neighbours of complex data objects and automatically assigning semantic class labels to them. It will use large scale, high dimensional, heterogeneous, incomplete, and noisy internet image labelling datasets as Case Study for technology development and evaluation. The FISH solutions and utilities can be directly applied to automatically label security surveillance videos and defence reconnaissance imageries to extract semantic meanings to facilitate security and defence intelligence gathering and analysis.

Defence reconnaissance systems such as the RAF's RAPTOR contain imaging sensors (visible and infrared) which can acquire huge volumes of high resolution imageries of ground targets. Rapidly and accurately interpreting the images, identifying and recognizing ground objects and assessing the battle-field situations will be very important. Post-battle analysis of these imageries will be also valuable for future operations. Manually analysing these large volumes of high resolution imageries can be both difficult and labour intensive. Automatic image analysis and interpretation will play a very important role in making good use of defence reconnaissance data such as those acquired by the RAF's RAPTOR. The research results of this project can be useful for these purposes.

The FISH algorithms and solutions can be applied to automatic labelling of the images and videos returned by RAPTOR. There are at least two scenarios where FISH can be useful. It can be implemented onboard the aircraft interpreting and labelling the images in real-time thus helping the pilots assessing the battle-field situations and react to them more rapidly. It can also be implemented in ground stations for post battle analysis and for archiving - by automatically labelling the images of different situations can facilitate search and retrieval of specific and relevant situations.

There are an estimated 1.85 million surveillance video cameras in the UK constantly collecting billions of image footages which could contain vital security information for antiterrorism, for preventing crime and for protecting human lives. For such huge volumes of data, it will be impossible to interpret them manually. Automatic tools will be valuable and FISH can help develop such tools.

The FISH algorithms and solutions can be used to automatically label security surveillance videos, identify dangerous and emergency situations quickly thus helping security authorities to gather security intelligence and react to them fast. Such techniques can be employed by the police and other security authorities for real-time situation monitoring or for post-event analysis - semantically labelled video frames can greatly facilitate the retrieval of specific and relevant contents.

For defence, the FISH technology can help extract meanings from imageries acquired by reconnaissance systems such as RAF's RAPTOR thus facilitating defence intelligence gathering. For security, the FISH technology can help extract meanings from security videos thus helping gathering security information which may be valuable for antiterrorism, preventing crime and saving lives.

In summary, the research of the FISH project can contribute state of the art technology for improving defence and security intelligence gathering and analysis which will lead to improvement in the UK's defence and security capability.

Beyond defence and security, the research of the FISH project will have wider applications in academic research, and in applications fields ranging from biomedical to multimedia and the internet.

In biomedical research, e.g., disease classification in medical images and general biomedical data classification and analysis such as matching DNA sequences involve finding similar data objects, the FISH technology can have a direct applications in these areas.