Phenomenology of Deep Learning

Recent Machine Learning (ML) breakthroughs in industry and the sciences rely on neural network architectures with multiple layers, a
branch of ML called Deep Learning (DL). Over the last decade, much of this progress was possible thanks to the impressive growth of
the size of datasets and neural networks. In contrast, the understanding of the foundations of DL has not followed this successful
trend. In fact, there is an enormous gap between the practical success of DL and our understanding of why DL works so well. It is
widely acknowledged that to expand the scope of ML applications and obtain reliable artificial intelligence systems, we must achieve
a fundamental understanding of DL.
A physics-based framework can provide a unique perspective to pressing questions in DL and contribute to filling the gap between
practical developments and theoretical foundations. We stress that, while ML applications are broadly used in physics, the flow of
ideas in the opposite direction, i.e., the use of concepts and techniques from theoretical physics to understand modern deep learning,
has only started to be explored.
In this MSCA, we will exploit and capitalize on the striking similarities between deep neural networks and effective field theory in
physics. The goal of "Phenomenology of Deep Learning" (PHENO-DL) is to contribute to the development of an effective theory of
deep learning. We will make use of physics model-building tools to attack foundational questions (e.g., how do deep neural networks
generalize?) and remarkable phenomena in deep learning (namely, double descent and adversarial examples). To this end, we will
adopt a setup inspired by established methods from theoretical physics, which have been recently applied to neural networks. In
particular, to explain the neural network's expressivity and capacity, we will use the interplay between the renormalisation group in
physics and the 'hierarchy of features' in the different layers in a deep neural network.