Classifying Images Regardless of Depictive Style

Computer's today can recognise objects in photographs. This ability has formed the basis of many familiar applications such as Facebook tagging, Google Image Search, Google Goggles, and automated passport checking at UK borders.

Yet a significant restriction remains: computers can only recognise objects in photographs. At least, their ability to recognise objects in
drawings and paintings - in artwork of any kind - is strictly limited. If this limitation can be overcome then many more applications will
become possible.

One is a new way to search the web for images in which a drawing (say) is dragged from the desktop into a search bar,
and paintings and photographs are given back to the user (at the moment a user gets the same sort of image back as was dragged into the
search bar).

Another is the automated production of catalogues for taxonomy - which is important to scientists faces with tens of thousands of
microscopic creatures; species catalogues are hand-drawn right now so automation would be a significant advance for them.

The output of the programme would also allow ordinary photographs to be converted to icons. This is not as dry as it sounds, but could help the visually impaired to gain access to photographic content. If photographs and drawings can be linked in the way this project has in mind, then
objects in photographs could be turned into icons rendered by a set of raised pins. So there would be a symbol for car, say, not unlike that which might be drawn by a child - and in fact this is very close to the icons blind artists draw. This would allow the visually impaired to read photographs in newspapers, or in text books, and allow them to share the holiday snaps of family and friends.

This proposal is about building the basic technology that underpins these applications, and quite possibly others too. Key to it is lifting the barrier that computers of today face - allowing them to recognise objects no matter how they are depicted.

The impact of this proposal will show in the near, medium, and longer terms.
It will also show in the academic sector, the industrial sector, in society, and is media friendly.
We plan to pursue impact in all of these.

The academic beneficiaries are detailed in another section of this form and so will not be detailed here,
expect to say that medium and longer impact requires further input from the academic communities.

Impact in the industrial sector has an obvious route via sophisticated search engines, as built by companies such as Google and Microsoft.
Internet search will benefit because the total of all images includes not just photographs but also artwork of all kinds.
The ability to pose queries as sketches and retrieve photographs is a current research topic; this proposal would not contribute directly
but we do plan to build a prototype as our application (objective 4). We note that Google offer PhD studentships, a route we intend to take advantage of as this proposal progresses, with a view to constructing a more substantive web-search application, and so we regard it as a medium term aim.

In terms of wider society, we find the possibility of allowing the visually impaired access to photographic content particularity appealing. In fact, because our philosophy is to not discriminate between depictive styles, our approach should allow the visually impaired access to a mush wider variety of images than at present. Access is possible if we can summarise visual content so as to remove unwanted clutter; recognising objects and synthesising icons that can be felt allows people to read newspapers, share photos, and generally raises their standard of living.
This would a medium to long-term aim.

The proposal is media friendly: the ability to drag a photograph of the Queen onto a search bar and have all of her portraits returned is something that would be of general interest. It is to the benefit of the research community as a whole and to research councils in particular to receive favourable publicity.