Cloud compression for live volumetric video
Lead Participant:
CONDENSE REALITY LTD
Abstract
Condense Reality has developed a system for capturing and streaming live volumetric video into Game Engines. This will allow customers to put on ticketed events in virtual spaces that can hugely expand the addressable market for live music.
To date the focus of Condense Reality has been live capture. It has solved many complex computer vision problems in order to get to this stage. This has been validated by a £3.45M VC investment in December 2021\.
It is now shifting focus to distribution and the delivery of immersive experiences to the end user. In order to do this there is a need to maintain high quality but reduce the bitrate of the live volumetric video that is streamed from a cloud based CDN. In order to continue to drive adoption there is a need to cater for customers with a wide range of connection speeds and on devices with varying processing capabilities. This project is focused on the novel application of deep learning to efficiently compress live-streamed volumetric video. This is a hard problem to solve.
University of Bristol is recognised as a world leader in AI-based video coding, with contributions to MPEG standards including new training databases, coding frameworks, and deep learning network architectures. In this collaborative project, researchers within the Visual Information Laboratory will contribute expertise and enhance their state-of-the-art coding approaches to volumetric video content.
**This project will focus on:**
1. Novel methods that employ deep learning to improve compression efficiency.
2. Enhanced architectures for delivering volumetric video content at variable bitrates.
3. Integrating systems into the decoder which dynamically influence the bitrate based on bandwidth and device capability (updating the Unity and Unreal plugins so that they can cope with whatever compression methodology is adopted).
Through this, we will provide the following **benefits to the user**:
1. Enhanced visual quality and immersive experience associated with volumetric video content
2. Reduced end-to-end latency of the compression systems
3. Improved unit economics of delivery
4. Increased content accessibility
This project will make a major contribution to the state of the art in volumetric video delivery and generate significant impact on a wide range of immersive video applications. Bristol has a unique ecosystem of researchers, technology companies and artists that can be at the forefront of defining and creating a future collaborative Metaverse. The world-leading innovations in this project will contribute to this ambition and thus bring with significant economic benefits to the region.
To date the focus of Condense Reality has been live capture. It has solved many complex computer vision problems in order to get to this stage. This has been validated by a £3.45M VC investment in December 2021\.
It is now shifting focus to distribution and the delivery of immersive experiences to the end user. In order to do this there is a need to maintain high quality but reduce the bitrate of the live volumetric video that is streamed from a cloud based CDN. In order to continue to drive adoption there is a need to cater for customers with a wide range of connection speeds and on devices with varying processing capabilities. This project is focused on the novel application of deep learning to efficiently compress live-streamed volumetric video. This is a hard problem to solve.
University of Bristol is recognised as a world leader in AI-based video coding, with contributions to MPEG standards including new training databases, coding frameworks, and deep learning network architectures. In this collaborative project, researchers within the Visual Information Laboratory will contribute expertise and enhance their state-of-the-art coding approaches to volumetric video content.
**This project will focus on:**
1. Novel methods that employ deep learning to improve compression efficiency.
2. Enhanced architectures for delivering volumetric video content at variable bitrates.
3. Integrating systems into the decoder which dynamically influence the bitrate based on bandwidth and device capability (updating the Unity and Unreal plugins so that they can cope with whatever compression methodology is adopted).
Through this, we will provide the following **benefits to the user**:
1. Enhanced visual quality and immersive experience associated with volumetric video content
2. Reduced end-to-end latency of the compression systems
3. Improved unit economics of delivery
4. Increased content accessibility
This project will make a major contribution to the state of the art in volumetric video delivery and generate significant impact on a wide range of immersive video applications. Bristol has a unique ecosystem of researchers, technology companies and artists that can be at the forefront of defining and creating a future collaborative Metaverse. The world-leading innovations in this project will contribute to this ambition and thus bring with significant economic benefits to the region.
Lead Participant | Project Cost | Grant Offer |
|---|---|---|
| CONDENSE REALITY LTD | £201,132 | £ 139,988 |
Participant |
||
| UNIVERSITY OF BRISTOL | £59,954 | £ 59,954 |
| UNIVERSITY OF BRISTOL |
People |
ORCID iD |
| Andy Littledale (Project Manager) |