Swarm Social Sound Modulators

The UK is a world-leader in creating interactive applications that are enabled by computational manipulation of acoustic wave fronts. Three examples of such applications include mid-air haptics, directional audio, and volumetric 3D particle displays. Using a phased array of ultrasonic speakers that are precisely and individually controlled, we can create high pressure focal points in mid-air. Modulating these focal points in various ways, it is possible to 1) create rich tactile sensations when touched by the bare hands, 2) steer directional sounds that propagate over long distances un-attenuated, 3) levitate small particles that when rapidly moved in space emulate volumetric 3D shapes due to a phenomenon called persistence of vision.

The exploitation of these amazing new technologies is uniquely available to Ultraleap (a UK based company) and Sussex University who have a long and productive history of collaboration. For example, Ultraleap is currently combining hand-tracking and ultrasonic mid-air haptic feedback solutions for applications ranging from VR training simulators, automotive interfaces, gaming machines, and next generation digital signage kiosks. Similarly, Sussex University is creating multimodal 3D displays based on rapidly updating ultrasonic phased arrays to create persistence of vision when moving acoustically levitated objects. A significant constraint to the wide-scale deployment of the underlying technology of phased arrays is the cost and complexity of a non-modular system because it limits applicability. For example, there is no one size fits all phased array with most integrated solutions needing to be custom developed.
In this project, we will circumvent such problems altogether by creating simple and low cost modular spatial sound modulator (SSM) units i.e. smaller arrays of acoustic sources, to be placed around the interactive space, that can collectively out-perform the single large monolithic solution we currently have. Moreover, we will take a leap forward in sound-field control by removing scalability and reusability issues thus opening up the exploitation of phased array technologies into other applications domains that can benefit from the non-contact delivery of haptic feedback, steerable directional sound, and/or volumetric 3D particle displays. Specifically, we will draw on the well-developed literature of multi-agent game theory and distributed computing and use them to build a decentralised swarm architecture that can flexibly accommodate numerous SSM units. Each SSM will emit and modulate the sound-field nearby it while sharing a common awareness of the contextual details with its swarm host, while the desired collective behaviour will emerge from the interactions between multiple SSMs and their interactions with the environment.

There are several anticipated benefits to our proposed approach. Firstly, by designing simple, independent SSMs we are able to address multiple commercial applications using the same primitive unit, while simultaneously streamlining the manufacturing pipeline. These modular units can be used individually or combined in a myriad of ways to create new applications. Secondly, by enabling a distributed control architecture, swarm SSMs can seamlessly and progressively scale up. By incrementally combining larger numbers of modular units, customers can initially use a small number of SSM units, and dynamically grow the capabilities of their interactive multimodal system by adding new devices according to the application needs. Finally, by using game theory we will enable SSMs to dynamically cooperate with each other as to meet application objectives independent of the application logic and the arrangement of our modular devices thus simplifying the development and design process and enabling creative designers to focus on the delivery of evermore immersive and multimodal experiences.