Visual Image Interpretation in Humans and Machines

The sense of vision is so fundamental to humans that it is a largely automated process which appears to us as extremely easy. This would suggest that it should be easy to make a computer see like a human. In fact this is a very difficult task because the biological visual system is very complex; occupying about one quarter of the human brain. Human vision is both highly effective and efficient. For example it is capable of identifying around 10,000 different object categories and can learn new categories from single examples. This in achieved with a system requiring just 20 watts of power and weighing 1.4kg. No computer system can match this performance for recognition ability, learning efficiency and power consumption.

One way to devise new computer vision methods is to understand how biological visual systems work. However, the complexity of vision has made this very difficult and some researchers have concentrated their efforts on understanding biological vision while others have sought independent solutions to specific problems in computer vision. For example, humans can read car number plates but we do so using a general purpose visual system that can also read gothic script and handwriting as well as performing a host of other tasks. Building a number plate recognition system to read letters in the same general way that humans do would be difficult. However, because number plates have a certain fixed format (they are always a certain, bright, colour, and the font is always a certain style and size) building a computer vision system just to read number plates, and nothing else, is a much simpler task.

There are some tasks that have not proved simple for computer vision and where understanding biological vision is likely to be essential to future success. One example is matching the appearance of two surfaces. Suppose you wanted to make artificial stone to look exactly like the real stones in a building. To get the recipe just right you would have to know not just the physical properties of the original stone (which probably cannot be matched exactly) but also how the human vision system is likely to perceive the stone. You can then pick a recipe that may not mimic the stone exactly but which will look just like the real stone to humans. Moreover, if you know how the visual system processes the colours and textures of surfaces you can build a computerised tool that can predict recipes automatically.

Another area of interest is computer graphics. One way to make computer graphics look convincing is to exactly model the physics of the thing you are trying to represent. However, such rendering methods are often very time consuming and computationally expensive. Because the human visual system does not see every detail in an object it is often possible to render graphics much more quickly and effectively using perceptual rendering techniques that exploit knowledge of how the human visual system will process each scene.

Because those researchers working on biological vision tend to be from Biology and Psychology backgrounds and those who research computer vision from Computer Science and Engineering backgrounds, there is often a gap in understanding between the two groups of researchers which makes it hard for them to work together on problems such as those outlined above. The aim of this Network is to bring such researchers closer together, both physically and scientifically, so that they can identify and work together on the challenging problems where success is most likely. We will achieve this by a series of away day style meetings and conferences and by funding junior scientists and PhD students to spend time working in another lab from a different discipline.

Network members
The impact of the Network will be felt directly in terms of training for the PhD students and early career researchers who will benefit from the exchange visits. There is also the wider learning and development of understanding that participating in the Network will bring, via workshop and conference participation, as well as the direct collaboration within projects.

Research Community
The research community will benefit from having a nexus of researchers spanning a number of EPSRC research areas who will be able to provide a better focus for cross-discipline research in terms of the ICT, and broader EPSRC, remit. Research areas where this benefit will be felt include: Image and Vision Computing, Vision Hearing and Other Senses, Graphics and Visualisation, Human Computer Interaction, Artificial Intelligence Technologies, Robotics, and Displays. The Network will provide a coherent portfolio of multi-disciplinary grants. The research community will also gain a more clearly defined group of researchers able to comment on the multi-disciplinary grants that the above themes tend to generate.

UK Economy, Healthcare Professionals, Education professionals, Schools and Museums, Public
These stakeholders will benefit from having an established and coherent researcher base in the area of biological and computer Vision. The Network will act as a facilitator for activity in this area and as such will not produce direct impact. Individual Network members will be responsible for translating and exploiting their own intellectual property. However, the potential application area for computer vision technology is huge raging from medical diagnostics and treatment to computer games and the film industry. We indicate some example application areas below but this list is by no means exhaustive:

Appearance technology has applications in the textile, paint and automotive industries where there is a desire to match the appearance of a synthetic product to some natural sample. There are applications of this technology too in retail where samples presented online need to match the physical product. Medical applications could include the construction of prosthetics. Designers, artists and education professionals will also benefit from the ability to produce physical replicas of artefacts or surface treatments that match natural samples.

Perceptual rendering (rendering to match the perceptual rather than physical properties of an object) has applications in computer graphics, computer games, and the film industry where more realistic rendering can be achieved more economically.

Object recognition technology might directly contribute to 'home help' style robots capable of finding and retrieving objects from around a standard house or able to perform simple household tasks. Many industrial settings can also benefit from systems capable of recognising objects in a range of poses and lighting conditions. Automated vehicles may benefit from object, place terrain and hazard recognition. Automatic face recognition technology has applications in security settings. Recognition applications for smart phone and wearable computers will help users to identify unusual objects, people and places and may also assist in the rehabilitation of stroke patients who often have specific memory deficits. Object recognition technology can also improve medical diagnosis.

Active vision for robots will enable them to better respond to changing environments to find and manipulate objects. Gesture and emotion recognition will have applications in the design of human computer interfaces and educational software. Recognition of emotions will also have applications in security where it may be possible to identify malicious intent. Emotion recognition applications for wearable computers may also help those with autism.