Scene Processing With Machine Learnable and Semantically Parametrized Representations RENEWAL

Generative artificial intelligence has made a significant leap in developing intuitive natural language based computer interfaces via large language models, and realistic image and video generation from text inputs. These have already started revolutionising creation and editing of code, text, images, videos, presentations, and many other digital media. We will extend these techniques to scene creation and processing by relying on the techniques we have been developing over the years. We will develop text-to-scene models for easy creation, intuitive control, and conversational capture of 3D objects and scenes. We will deploy these models to the new generation of extended reality devices with advanced displays, optics, sensors, and processing units to build a scene creation and editing system that uses voice and brain signals as inputs.