BBC Prosperity Partnership: Future Personalised Object-Based Media Experiences Delivered at Scale Anywhere

Lead Research Organisation: University of Surrey
Department Name: Vision Speech and Signal Proc CVSSP

Abstract

Personalisation of media experiences for the individual is vital for audience engagement of young and old, allowing more meaningful encounters tailored to their interest, making them part of the story, and increasing accessibility. The goal of the BBC Prosperity Partnership is to realise a transformation to future personalised content creation and delivery at scale for the public at home or on the move.

Evolution of mass-media audio-visual 'broadcast' content (news, sports, music, drama) has moved increasingly towards Internet delivery, which creates exciting potential for hyper-personalised media experiences delivered at scale to mass audiences. This radical new user-centred approach to media creation and delivery has the potential to disrupt the media landscape by directly engaging individuals at the centre of their experience, rather than predefining the content as with existing media formats (radio, TV, film). This will allow a new form of user-centred media experience which dynamically adapts to the individual, their location, the media content and producer storytelling intent, together with the platform/device and the network/compute resources available for rendering the content.The BBC Prosperity Partnership will position the BBC at the forefront of this 'Personalised Media' revolution enabling the creation and delivery of new services, and positioning the UK creative industry to lead future personalised media creation and intelligent network distribution to render personalised experiences for everyone anywhere.

Realisation of personalised experiences at scale presents three fundamental research challenges: capture of object-based representations of the content to enable dynamic adaption for personalisation at the point of rendering; production to create personalised experiences which enhance the perceived quality of experience for each user; and delivery at scale with intelligent utilisation of the available network, edge and device resources for mass audiences. The BBC Prosperity Partnership will address the major technical and creative challenges to delivering user-centred personalised audience experiences at scale. Advances in audio-visual AI for machine understanding of captured content will enable the automatic transformation of captured 2D video streams to an object-based media (OBM) representation. OBM will allow adaptation for efficient production, delivery and personalisation of the media experience whilst maintaining the perceived quality of the captured video content. To deliver personalised experiences to audiences of millions requires transformation of media processing and distribution architectures into a hybrid and distributed low-latency computation platform, allowing flexible deployment of compute-intensive tasks across the network. This will achieve efficiency in terms of cost and energy use, while providing optimal quality of experience for the audience within the technical constraints of the system.
 
Description Frameworks for Understanding Personalisation - BBC White Paper WHP 404
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to new or improved professional practice
Impact Future of media services
URL http://www.bbc.co.uk/rd/pubs/whp
 
Description Ofcom Object-based Media Working Group
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
Impact Influence on media communication and service regulation
URL https://www.ofcom.org.uk
 
Description The Role of the Audience in Media - BBC White Paper WHP 396
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to new or improved professional practice
Impact White paper on future media
URL https://www.bbc.co.uk/rd/publications/role-of-audience-in-media-how-culture-framing-narration-shape-...
 
Title Audio-visual production dataset for weather forecaster personalisation use case 
Description Audio and video recording of presenters for production resarch 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact Use in production demonstrators January 2023 
URL http://cvssp.org
 
Description AudioScenic 
Organisation Audioscenic
Country United Kingdom 
Sector Private 
PI Contribution Advisor/collaboration on spatial audio research and development for personalised media
Collaborator Contribution Participation in industry events
Impact Research advice in spatial audio
Start Year 2021
 
Description BBC Research and Development 
Organisation British Broadcasting Corporation (BBC)
Country United Kingdom 
Sector Public 
PI Contribution Research in Computer Vision for broadcast production and Audio. Technologies for 3D production, free-view point video in sports, stereo production from monocular cameras, video annotation Member of the BBC Audio Research Partnership - developing the next generation of broadcast technology.
Collaborator Contribution In kind contribution (members of Steering/Advisory Boards) Use of the BBC lab and research/development facilities. Studentships (industrial case) funding and co-supervision of PhD students.
Impact Multi-disciplinary collaboration involves Computer Vision, Video Analysis, Psychoacoustics, Signal Processing and Spatial Audio
 
Description BT 
Organisation BT Group
Country United Kingdom 
Sector Private 
PI Contribution research collaboration in object-based/personalised media delivery
Collaborator Contribution support of research activities
Impact ongoing collaboration
Start Year 2021
 
Description British Broadcasting Corporation 
Organisation British Broadcasting Corporation (BBC)
Country United Kingdom 
Sector Public 
Start Year 2004
 
Description Dimension Studios 
Organisation Dimensional Imaging Limited
Country United Kingdom 
Sector Private 
PI Contribution Access to expertise and data for volumetric capture
Collaborator Contribution Collaboration partner in volumetric performance capture
Impact Research collaboration
Start Year 2021
 
Description Figment Productions 
Organisation Figment Productions Limited
Country United Kingdom 
Sector Private 
PI Contribution Advice/collaboration on personalisation in XR experiences
Collaborator Contribution research collaboration and attendance at industry meetings
Impact ongoing research
Start Year 2021
 
Description Foundry 
Organisation Foundry
Country United Kingdom 
Sector Private 
PI Contribution Collaboration on Computer Vision and AI tools for film and VFX production
Collaborator Contribution Novel computer vision methods for segmentation, reconstruction, tracking and representation of people from video. Used for actor performance capture and visual effects production.
Impact Novel Computer vision methods for video analysis of actor performance
Start Year 2008
 
Description Foundry 
Organisation Foundry
Country United Kingdom 
Sector Private 
PI Contribution Collaboration on Computer Vision and AI tools for film and VFX production
Collaborator Contribution Novel computer vision methods for segmentation, reconstruction, tracking and representation of people from video. Used for actor performance capture and visual effects production.
Impact Novel Computer vision methods for video analysis of actor performance
Start Year 2008
 
Description Imagination Technologies 
Organisation Imagination Technologies
Country United Kingdom 
Sector Private 
PI Contribution Contribution to advise on personalisation for the mobile market
Collaborator Contribution Industry presentation and collaboration events
Impact Ongoing collaboration
Start Year 2021
 
Description Imagineer Systems 
Organisation Boris FX
Country United Kingdom 
Sector Private 
PI Contribution Personalised media production tools
Collaborator Contribution Advice/collaboration on production tools
Impact ongoing collaboration
Start Year 2021
 
Description Intel 
Organisation Intel Corporation
Country United States 
Sector Private 
PI Contribution Research collaboration in personalised media
Collaborator Contribution collaboration/PhD Sponsorship
Impact ongoing collaboration
Start Year 2021
 
Description Mirriad 
Organisation Mirriad
Country United Kingdom 
Sector Private 
PI Contribution Personalisation in advertising advice
Collaborator Contribution Industry partner briefing on requirements for personalisation
Impact Ongoing collaboration
Start Year 2021
 
Description Sony Broadcast and Professional Europe 
Organisation SONY
Department Sony Broadcast and Professional Europe
Country United Kingdom 
Sector Private 
Start Year 2004
 
Description Telefonica 
Organisation Telefonica S.A
Department Telefonica Research
Country Spain 
Sector Private 
PI Contribution Research collaboration in personalised/object-based media
Collaborator Contribution media delivery
Impact ongoing research
Start Year 2021
 
Description BBC Sounds Amazing 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Industry/academic forum for research and production industry professionals in audio and sound hosted by the BBC
Year(s) Of Engagement Activity 2021,2022
URL https://www.bbc.co.uk/academy/events/sounds-amazing-2022/
 
Description BBC Wing Watch public trial 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact If so you can you could include the public trial of the Wing Watch service during Winterwatch
https://www.bbc.co.uk/rd/blog/2023-01-wing-watch-ai-bird-wildlife-video

This allowed us to experiment with how AI-generated data can be used to drive a flexible media experience, giving the user the freedom to navigate the media in the way that most interest then. In doing so we were also experimenting with production tools to manage that AI-data to make sure it was suitable for presenting to the audience.
Year(s) Of Engagement Activity 2022,2023
URL https://www.bbc.co.uk/rd/blog/2023-01-wing-watch-ai-bird-wildlife-video
 
Description CVPR 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Primary international forum for computer vision and AI/machine learning research in audio-visual media. Research dissemination through papers, key-note invited talks and workshop organisation
Year(s) Of Engagement Activity 2021,2022,2023
URL https://cvpr2023.thecvf.com
 
Description European Conference on Visual Media Production 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation of research advances at industry-academic forum
Year(s) Of Engagement Activity 2021,2022
URL https://www.cvmp-conference.org/