Theoretical Properties of Diffusion Models

Score-matching generative models, particularly diffusion models, have emerged as a leading approach in generative modeling. These models have achieved state-of-the-art performance across a range of domains, including audio, image, and video synthesis, as well as specialized applications like molecular modeling and, more recently, text generation. Their success has sparked interest in better understanding the theoretical properties that contribute to their effectiveness.
Iteration Complexity
A central question in the study of diffusion models is determining their iteration complexity-the number of steps required for the model to converge to an accurate approximation of the target distribution. Reducing iteration complexity is crucial for improving the efficiency of these models, as each step entails significant computational cost. Peter's research focuses on exploring iteration complexity within the framework of the manifold hypothesis, which suggests that high-dimensional data (e.g. images or molecular structures) often lie on a lower-dimensional manifold within the larger ambient space. If this hypothesis holds, diffusion models may require a number of steps proportional to the dimension of this manifold, rather than the full high-dimensional space, thereby potentially reducing the number of required steps.
Generalization Properties
Another significant area of research involves the generalization capabilities of diffusion models-their ability to generate high-quality samples that they have not encountered during training. Diffusion models have exhibited strong generalization across various domains, though the mechanisms behind this are not yet fully understood. Peter is particularly interested in studying these properties in the context of the manifold hypothesis. Explaining why diffusion models generalize well could offer insights for improving future versions of these models.