Equality Saturation for Deep Learning Compilers

Exponential growth in the computational demands of state-of-the-art Deep Learning (DL) models poses both societal and
environmental concerns as research is increasingly monopolized by corporate labs which can afford multi-million dollar
training processes which in turn produce tens of thousands of kilograms of CO2. Improving the efficiency of DL is
therefore paramount to democratizing research and protecting the planet. Equality Saturation, a novel compiler
optimization technique, is one promising approach, reducing DL runtime by up to 70% and finding such improvements
300x faster than traditional methods in some scenarios. However, its widespread application to Deep Learning Compilers
is limited by its complexity, scalability and the difficulty in adapting it to the heterogeneous DL frameworks available
today. By integrating Equality Saturation into the MLIR compiler, Equality Saturation could be widely applied across
frameworks without requiring tens of thousands of lines of handwritten optimization code, thereby increasing its
accessibility to developers and greatly improving the efficiency of modern DL systems.