Statistically explainable GAN inversion

Generative adversarial networks (GANs) are a deep learning framework that learns a mapping, from a latent space to a data space, to generate new data points (often images) with the same probability distribution as that of the training data. Its reverse dual, GAN inversion, aims to invert a given image back into the latent space of a pretrained GAN model, such that editing of an image can be simply achieved by editing its code in the latent space. However, to achieve desired image editing (e.g. style transfer, imagination realisation or fine-grained modification), it is pivotal to understand, model and infer the statistical structure and randomness of the latent space of a pretrained GAN model. This is the aim of this project, a new and interdisciplinary topic able to inspire numerous exciting image-editing innovations and applications. To achieve the aim, the student will investigate from three perspectives: firstly to discover independent latent factors of the desired attributes by exploring nonlinear dimension reduction of the latent space; secondly to model interpretable fine-grained controls with some intermediate priors from domain knowledge as regularisation; and finally to infer from the latent space a common subspace for multimodal synchronisation between images, text and audios.

Research Areas:
Artificial intelligence technologies Statistics and applied probability