Illuminating the Path of Video Visualization

Lead Research Organisation: University of Oxford
Department Name: Oxford e-Research Centre

Abstract

The notion of video visualization was coined by the PI and his postgraduate student in their 2003 IEEE VIS paper. It is a technology drawing the concepts and methodologies from volume and flow visualization, image and video processing, and vision science. It extracts meaningful information from a video data set and conveys the extracted information to users in appropriate visual representations. It is not intended to provide fully automatic solutions to the traditional problems in video processing, but involves human in the loop of intelligent reasoning while reducing the burden of viewing videos. In the subsequent work in collaboration with Stuttgart, the PI and CI introduced the concept of visual signatures in video visualization, and reported a major user study conducted at Swansea involving some 92 subjects [IEEE TVCG 2006]. This work offered an important scientific insight as to how human observers may learn to recognize visual signatures of events depicted in an abstract visual representation of a video.[Tsotsos01] stated that a bounded visual search (e.g., looking for all moving pixel clusters with 20-60 pixels) can be achieved in linear time, whist an unbounded visual search (e.g., looking for something abnormal in a video) is NP-complete. For most practical problems in video processing and computer vision, we rarely have perfectly bounded visual search. We often search simultaneously for entities (e.g., objects, motions or events) in different classes. The models that are used to guide a search are usually incomplete and may lead to uncertainty or errors in detection, segmentation andclassification. The dynamic and unpredictable nature of the input videos instigates mechanisms for heuristic reasoning and iterative decision optimization, which further depart from linear or polynomial performance.In contrast, the human eye-brain system is undeniable more powerful than any current vision system in performing visual searches, especially unbounded visual searches. Even we suppose that the human eye-brain system is a Turing machine, its 100 billion neurons and 100-500 trillion synaptic connections between neurons will unlikely to be matched by computers in the near future. Hence this raises the possibility that using video visualization to aid unbound visual search may provide a more scalable means for dealing with large volumes of video datasets.Video visualization can be deployed in many application areas, such as scientific experimentation and computation, security industry and media and entertainment industry. However, in traditional visualization (e.g., medical visualization), the users are normally familiar with the 3D objects (e.g., bones or organs) depicted in a visual representation. In contrast, human observers are not familiar with the 3D objects depicted in a visual representation of a video because one spatial dimension of these objects shows the temporal dimension. The problem is further complicated by the fact that, in most videos, each 2D frame is the projective view of a 3D scene. Hence, a visual representation of a video on a computer display is, in effect, a 2D projective view of a 4D spatiotemporal domain. In order to for us to see 'time' without using 'time', we need to address a range of challenges in science, technology, visual perception and applications. This project is intended to continue the UK's leadership in tackling these challenges by building on the existing expertise and excellence in video visualization.

Publications

10 25 50
publication icon
Borgo R (2012) An Empirical Study on Using Visual Embellishments in Visualization. in IEEE transactions on visualization and computer graphics

publication icon
Chung D (2013) Glyph sorting: Interactive visualization for multi-dimensional data in Information Visualization

publication icon
Chung D (2014) Visualizing multiple error-sensitivity fields for single camera positioning in Computing and Visualization in Science

publication icon
Chung DH (2016) Knowledge-Assisted Ranking: A Visual Analytic Application for Sports Event Data. in IEEE computer graphics and applications

publication icon
Duffy B (2015) Glyph-Based Video Visualization for Semen Analysis. in IEEE transactions on visualization and computer graphics

publication icon
Fang H (2013) Visualizing natural image statistics. in IEEE transactions on visualization and computer graphics

publication icon
Legg Philip A. (2013) Transformation of an Uncertain Video Search Pipeline to a Sketch-Based Visual Analytics Loop in IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS

Related Projects

Project Reference Relationship Related To Start End Award Value
EP/G006555/1 01/02/2009 29/06/2011 £843,148
EP/G006555/2 Transfer EP/G006555/1 29/06/2011 28/09/2013 £371,464
 
Description Summarizing a video into a single picture can significantly save the time of viewing videos in many applications. This research has proven the feasibility through a number of applications including sports (snooker, rugby), biological videos, traffic videos, satellite images, movie sound annotation, and so on.
Exploitation Route This project has resulted over 20 publications, including 17 journal papers. Two PDRAs who worked on the project have become faculty members. Video visualization is becoming a subject in visualization, and sports video visualization became a popular topic in 2014.
Sectors Aerospace, Defence and Marine,Construction,Digital/Communication/Information Technologies (including Software),Environment,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology,Security and Diplomacy,Transport

URL http://www.oerc.ox.ac.uk/projects/illuminating-path-video-visualization
 
Description This research [2009-2014] has generated a large collection of research results including a major theoretical paper as well as several applications (sports, biology transport and security). Over 20 papers (including 17 journal papers) resulted from this project.
First Year Of Impact 2010
Sector Leisure Activities, including Sports, Recreation and Tourism,Pharmaceuticals and Medical Biotechnology,Security and Diplomacy