Visual Image Interpretation in Humans and Machines

Lead Research Organisation: University of Bath
Department Name: Computer Science

Abstract

The sense of vision is so fundamental to humans that it is a largely automated process which appears to us as extremely easy. This would suggest that it should be easy to make a computer see like a human. In fact this is a very difficult task because the biological visual system is very complex; occupying about one quarter of the human brain. Human vision is both highly effective and efficient. For example it is capable of identifying around 10,000 different object categories and can learn new categories from single examples. This in achieved with a system requiring just 20 watts of power and weighing 1.4kg. No computer system can match this performance for recognition ability, learning efficiency and power consumption.

One way to devise new computer vision methods is to understand how biological visual systems work. However, the complexity of vision has made this very difficult and some researchers have concentrated their efforts on understanding biological vision while others have sought independent solutions to specific problems in computer vision. For example, humans can read car number plates but we do so using a general purpose visual system that can also read gothic script and handwriting as well as performing a host of other tasks. Building a number plate recognition system to read letters in the same general way that humans do would be difficult. However, because number plates have a certain fixed format (they are always a certain, bright, colour, and the font is always a certain style and size) building a computer vision system just to read number plates, and nothing else, is a much simpler task.

There are some tasks that have not proved simple for computer vision and where understanding biological vision is likely to be essential to future success. One example is matching the appearance of two surfaces. Suppose you wanted to make artificial stone to look exactly like the real stones in a building. To get the recipe just right you would have to know not just the physical properties of the original stone (which probably cannot be matched exactly) but also how the human vision system is likely to perceive the stone. You can then pick a recipe that may not mimic the stone exactly but which will look just like the real stone to humans. Moreover, if you know how the visual system processes the colours and textures of surfaces you can build a computerised tool that can predict recipes automatically.

Another area of interest is computer graphics. One way to make computer graphics look convincing is to exactly model the physics of the thing you are trying to represent. However, such rendering methods are often very time consuming and computationally expensive. Because the human visual system does not see every detail in an object it is often possible to render graphics much more quickly and effectively using perceptual rendering techniques that exploit knowledge of how the human visual system will process each scene.

Because those researchers working on biological vision tend to be from Biology and Psychology backgrounds and those who research computer vision from Computer Science and Engineering backgrounds, there is often a gap in understanding between the two groups of researchers which makes it hard for them to work together on problems such as those outlined above. The aim of this Network is to bring such researchers closer together, both physically and scientifically, so that they can identify and work together on the challenging problems where success is most likely. We will achieve this by a series of away day style meetings and conferences and by funding junior scientists and PhD students to spend time working in another lab from a different discipline.

Publications

10 25 50
 
Description ViiHM is a network of researchers who are primarily based in the UK. The network's purpose is to foster dissemination and collaboration between the Computer and Biological Vision communities. A primary part of ViiHM for this is it's annual event, and this has been the catalyst for a range of activities such as mini-workshops and exchange visits. These provide focused mechanisms for dissemination/discussion but also opportunities to discuss research projects and further funding. ViiHM has successfully fostered a range of these activities, for example:
Mini-workshops
- Binocular Vision, Challenges and Techniques: Ross Goutcher
- Vision for movement, a comparative approach: Paul Graham
- Statistics in Human and Machine Vision: Joshua Solomom
- Advancements in Face Research: A Conversation Between Humans and Machines:Yukun Lai
- Higher level scene understanding and application in robotic vision: Deepayan Bhowmik
- Young Vision Research Colloquium: Dave Bull
- The Social Impact of Facial Appearance: Moi Hoon Yap
- Low power and adaptive computer vision for scene understanding: Deepayan Bhowmik
Grant writing workshops
- Dynamic Faces: Representations for Animation and Perception & Bio-inspired models of gaze, action and reward for robotics: Alan Johnston, Darren Cosker & Iain Gilchrist
- Visual, thermal, near-­infrared, chemical and perceptual analysis of human skin for application in health, cosmetics and rendering: Marina Bloj
- The role of chromatic signals in spatial vision: Jasna Martinovic
- Illumination in Natural Scenes: Andrew Schofield
- Exploring Gaze Dynamics for Human-Machine Interactions: Farzin Deravi
- Retina Models: Sonya Coleman
- How gaze behaviour contributes to individual differences in cognitive task performance - scanpath analysis: Jinghao Xue & Kun Guo
- Biological Vision Inspired Dynamic 3D Acquisition and Reconstruction using Statistical Machine Learning: Yukun Lai
Exchange visits
- Perceived gloss of HDR stimuli (Wendy Adams, Psychology @ Southampton -> Rafal Mantiuk, Computer Science @ Bangor)
- Exploration of optimal gaze pattern in eye tracking studies using deep learning approach and its application to automatic image retrieval and object detection (Ruixuan Wang, Engineering & Physical Sciences @ Heriot-Watt -> Kun Guo, Psychology @ Lincoln)
- Active Vision with Human-in-the-Loop for the Visually Impaired ( Mr Jacobus Lock PhD student,  School of Computer Science, University of Lincoln -> Professor Iain Gilchrest University, School of Experimental Psychology)
Exploitation Route As ViiHM is focused on research culture, it has fostered new collaborations and partnerships. These are arguably as a result of ViiHM. The Birmingham PI - Andrew Schofield - will have further details.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Pharmaceuticals and Medical Biotechnology,Security and Diplomacy

URL http://www.viihm.org.uk/home/events/exchanges/
 
Description Annual ViiHM Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The workshop brought together members of the ViiHM network to share research results and discuss collaboration.
Year(s) Of Engagement Activity 2015
URL http://www.viihm.org.uk/home/events/second-workshop/
 
Description Annual ViiHM Workshop - 2014 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The ViiHM Annual meeting ran in 2014, in Stratford -upon-Avon. The event was attended by national and international academics, who networked and presented work. From this meeting, micro-workshops were organised by the community, resulting in grant applications being generated. At least one has been submitted to EPSRC.
Year(s) Of Engagement Activity 2014
URL http://www.viihm.org.uk/home/events/first-workshop/
 
Description Knowledge Transfer Day 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Members of CAMERA from Health & Computer Science attended a day-long workshop - talks from different department, supporting cross-faculty collaborations.
Year(s) Of Engagement Activity 2018
 
Description Mini-Workshops and Exchanges 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact As part of ViiHM, mini-workshops and grant writing events were organised by the community. The following exchanges/workshops have taken place so far:
Mini-workshops

- Binocular Vision, Challenges and Techniques: Ross Goutcher
- Vision for movement, a comparative approach: Paul Graham
- Statistics in Human and Machine Vision: Joshua Solomom

Grant writing workshops

- Dynamic Faces: Representations for Animation and Perception & Bio-inspired models of gaze, action and reward for robotics: Alan Johnston, Darren Cosker & Iain Gilchrist
- Visual, thermal, near-­infrared, chemical and perceptual analysis of human skin for application in health, cosmetics and rendering: Marina Bloj
- The role of chromatic signals in spatial vision: Jasna Martinovic
- Illumination in Natural Scenes: Andrew Schofield
- Exploring Gaze Dynamics for Human-Machine Interactions: Farzin Deravi

Exchange visits

- Perceived gloss of HDR stimuli (Psychology @ Southampton -> Computer Science @ Bangor): Wendy Adams
- Exploration of optimal gaze pattern in eye tracking studies using deep learning approach and its application to automatic image retrieval and object detection (Engineering & Physical Sciences @ Heriot-Watt -> Psychology @ Lincoln): Ruixuan Wang
Year(s) Of Engagement Activity 2014,2015
URL http://www.viihm.org.uk/home/events/exchanges/
 
Description University-Industry R&D Collaborations in Immersive Technologies Workshops 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact CAMERA organised two workshops with Immerse UK: University-Industry R&D Collaborations in Immersive Technologies. Secrets to Winning Innovation Funding and Running Successful Collaborative Projects in Partnership with Universities. Immerse UK and CAMERA teamed up to create two workshops aimed at catalysing new collaborations between UK Universities and innovative small businesses.

The theme of the workshops was leveraging cross-sector partnerships and enabling, immersive technologies to stimulate collaborative R&D projects for innovation and business growth. The workshops drew on the collective experience of collaborators from industry and academia, alongside funding providers and bid writers. Participants discussed what is involved in building successful collaborations with universities, what funding routes are available in relevant technology areas, and how best to access them. Participants left the workshops with a better understanding of the University research offering, collaboration routes, potential partnerships, relevant identified funding calls, and possibly a new project.

Thursday 3rd May 2018 - Bath The Innovation Centre, Broad Quay, Bath BA1 1UD

Wednesday 23rd May 2018 - London University of Bath, Level 6, 83 Pall Mall, St James's, London SW1Y

CAMERA were delighted to be asked to run two fully funded workshops for micro and SME companies focusing on successful University-Industry R&D collaborations in Immersive Technologies. We were pleased to welcome a wide range of SMEs, ranging from the fashion industry to documentary teams and one person start-ups. The two sold-out workshops drew on the collective experience of seasoned collaborators from industry and academia, including Marshmallowlaserfeast's Nell Whitley and Professor Adrian Hilton of the University of Surrey.

Delegates also heard from funding providers and experienced bid writers. Claire Tansley and Carrie Wootten gave valuable insights into national research funding and innovation networks, and Wavecrest's Jeff Clifford (former Head of R&D at Double Negative) took us through the most common types of collaborative R&D public grant funding and application processes.

Delegates also had the opportunity to informally discuss a variety of research areas with academics from various universities including Bath, Bath Spa, Ravensbourne, Surrey and Greenwich. Thanks also to Setsquared's Miles Davis for talking about the new Scale Up Programme, and University of Bath's Rosie Bennet, who took us through the new ERDF funded programmes on offer for digital technology businesses in the South West.

Attendees left with better insight into the university research offering, collaboration routes, potential partnerships, relevant identified funding calls, and even some new projects. We wish all our participants success and hope to work with many of them in the future.
Year(s) Of Engagement Activity 2018
 
Description VIIHM 2016 Annual Workshop - leading to mini-workshops, grant submissions and exchanges. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact As part of the ViiHM network, annual events are held. These include invited submissions/talks/posters from the Human and Computer Vision communities, and invited keynotes. The reach of keynotes and participants is international (although registered attendees are mostly national).
The annual workshops are the flagship ViiHM event, and are the catalyst for other activities - including mini-workshops and exchanges.
Listed below are some the outcomes where the annual event is arguably the catalyst:
Mini-workshops
- Binocular Vision, Challenges and Techniques: Ross Goutcher
- Vision for movement, a comparative approach: Paul Graham
- Statistics in Human and Machine Vision: Joshua Solomom
- Advancements in Face Research: A Conversation Between Humans and Machines:Yukun Lai
- Higher level scene understanding and application in robotic vision: Deepayan Bhowmik
- Young Vision Research Colloquium: Dave Bull
- The Social Impact of Facial Appearance: Moi Hoon Yap
- Low power and adaptive computer vision for scene understanding: Deepayan Bhowmik
Grant writing workshops
- Dynamic Faces: Representations for Animation and Perception & Bio-inspired models of gaze, action and reward for robotics: Alan Johnston, Darren Cosker & Iain Gilchrist
- Visual, thermal, near-­infrared, chemical and perceptual analysis of human skin for application in health, cosmetics and rendering: Marina Bloj
- The role of chromatic signals in spatial vision: Jasna Martinovic
- Illumination in Natural Scenes: Andrew Schofield
- Exploring Gaze Dynamics for Human-Machine Interactions: Farzin Deravi
- Retina Models: Sonya Coleman
- How gaze behaviour contributes to individual differences in cognitive task performance - scanpath analysis: Jinghao Xue & Kun Guo
- Biological Vision Inspired Dynamic 3D Acquisition and Reconstruction using Statistical Machine Learning: Yukun Lai
Exchange visits
- Perceived gloss of HDR stimuli (Wendy Adams, Psychology @ Southampton -> Rafal Mantiuk, Computer Science @ Bangor)
- Exploration of optimal gaze pattern in eye tracking studies using deep learning approach and its application to automatic image retrieval and object detection (Ruixuan Wang, Engineering & Physical Sciences @ Heriot-Watt -> Kun Guo, Psychology @ Lincoln)
- Active Vision with Human-in-the-Loop for the Visually Impaired ( Mr Jacobus Lock PhD student,  School of Computer Science, University of Lincoln -> Professor Iain Gilchrest University, School of Experimental Psychology)
Year(s) Of Engagement Activity 2016
URL http://www.viihm.org.uk/home/events/exchanges/
 
Description Workshop Organisation 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Dario Cazzola was personally invited to organise a workshop and give seminars to the OpenSim Workshop in Leuven (Belgium) - 6-8/11/2018

The Human Movement Biomechanics research group along with OpenSim fellows and experts (Luca Modenese from Imperial College, Colin Smith from ETH, and Dario Cazzola from the University of Bath) held an OpenSim workshop in Leuven.
Year(s) Of Engagement Activity 2018
URL https://gbiomed.kuleuven.be/english/research/50000737/groups/HMB/events/Opensimworkshop2018/opensimw...