Sound, Image and the Brain: Cognitive Live-Arts Technology in Contemporary Game-Oriented and Accessibility Paradigms.

As a result of the ever-decreasing cost and increasing efficiency of contemporary consumer hardware and software, advanced audiovisual technologies are now commonplace, even in mobile telecommunications devices. Carefully designed products and services, such as the Netbook and the iPhone, exemplify modern transformations in the contemporary technoculture, providing not just new technology, but more importantly, new economic and social infrastructures by which we engage with audiovisual experience and beyond. These devices continue to offer fresh opportunities for and challenges to the way we perceive our everyday life, augmenting our understanding of the world around us, by delivering information to us at high speeds, and in increasingly intuitive ways.

This is to some extent enabled by developments in human computer interface research. Interface technologies that until now have been too expensive or too complex for mass production are being prototyped and deployed as part of new consumer devices. Amongst these devices are touch screen or 'surface' interfaces (of which the iPhone is the key example), miniature camera / projection based systems that deploy computer vision and machine listening, such as Microsoft's project Natal and MIT's Sixth Sense, and consumer grade brain-computer interfaces, such as Emotiv systems Epoch, and Neurosky's Mindset Electroencephalograph device.

These developments have captured the imagination of the public, perpetuating a narrative of the technological extension of humanity. Researchers exploring new directions for interface technology support this extension through terminology such as 'augmented reality' and 'sixth sense technology'. Importantly, what this revolution also brings is the capacity for technology to enhance the lives of those whose experience of everyday life is very different to that of the majority, augmenting sensory experience and capacity in an interesting and highly fertile way. Most significantly, rather than restricting the potential dissemination routes for the application of these technologies, collaboration with those whose sensory experience is different and/or unique continues to add huge value and impact to the refinement of these approaches, allowing lessons learned in less common situations to be deployed in more general scenarios, and vice versa. For example, brain-computer interface technology is of great potential value to those with limited mobility, cognitive impairment or other form of disability, and its use in these environments will continue to reveal the extent to which it is becoming generally applicable.

Through a multidisciplinary approach that draws on perception and cognition, media engineering, therapy, interactive gaming, sound, music and audiovisual arts, this proposal aims to take completed research in brain-computer interfaces, audio-visualisation, participation and gaming, and develop it in partnership with industry and public organisations in order that it might reach its full potential in terms of social and economic impact, by engaging more fully with those within the public sector who both stand to benefit from, and also contribute to the creation and enhancement of consumer-grade real-time interaction hardware and software for brain-computer interfacing and technology-led creativity.

This proposed project has the potential for large public impact in both specific social, cultural and community arenas, and also in the mainstream economic sector. Given the main project partner is a limited company, and not a public organisation, its existence depends on continuing economic success. In addition, given the proposed Europe-wide exclusivity with respect to the Neurosky headset hardware, and that this device has only three technological competitors in the new marketplace, it is likely that this project will achieve high level public impact, not just in the UK, but all over the EU. As software developed through this project will be bundled directly with hardware (retailing for approx. £150 - £200), it will be well within the price range of ordinary consumers. In addition, any other party in Europe wishing to acquire and use the headset for any other purpose will receive bundled software developed as part of the project. This guarantees significant potential impact to a large number of potential stakeholders and observers alike. Furthermore, as Neurosky are a US company, economic and social impact is likely to extend to US territories also.

Crucially, as the project seeks to engage in the first instance with specific user groups through public sector partnership, the project stands to deliver significant social impact to the participating communities. We hope to be able to offer genuine improvements in the quality and experience of disabled user-group's engagement with sound, music, visual media and communications through effective real-world applications based on research in the electronic and technology arts sector. These improvements will be based on real-world user cases and testing, organised and ethically monitored by professional public bodies with years of expertise in the area of public engagement around technology arts in the disabled sectors. These workshops, to be organised in collaboration with qualified health professionals, will provide fine grained feedback on the responsiveness, ease of use and effectiveness of both visualisation based and EEG based therapeutic tools. These tools will then be given the maximum possible opportunity to impact upon the everyday lives and experience of the community user groups involved.

If successful, it is likely that this project will generate significant public attention in the mainstream press and even broadcast. This is supported by the already relatively high level public impact of the research project which featured on BBCi news (where it featured on the front page for several days), and BBC TV (including BBC's premiere TV program for the deaf and hard of hearing, 'See Hear', and 'The Gadget Show' (Roll7)), radio interviews including BBC World, BBC Radio 3, BBC World Service (radio), Russian TV, Discovery Channel Canada and Australian national radio among others. The work continues to be widely discussed on the Internet and in international press.

This demonstrates the size and importance of the potential genuine human interest in and around this project. Finally, it is expected that the combined social and economic impact of this proposed knowledge transfer fellowship has the capacity to dwarf the significant impact already represented by the research upon which it is based.