Continuous-time optimization on Riemannian manifolds and applications to deep learning

The PhD project focuses on the optimization of functions over Riemannian manifolds, specifically targeting the manifold of low-rank matrices. This manifold is characterized as the Cartesian product of riemannian manifolds. A primary application of this research is the low-rank training of neural networks, which has significant implications for enhancing computational efficiency and reducing memory usage. Central to the analysis within this project is the formulation of a continuous-time Ordinary Differential Equation that models the optimization process. The dynamics of this ODE enable a comprehensive analysis, facilitating the understanding of convergence properties and the behavior of the optimization trajectories in the manifold space. This in-depth study of the ODE's properties is crucial for developing efficient algorithms for training and deploying neural networks.