Neural network topologies from a hardware perspective

The challenges of deploying overparameterized neural networks (NNs) on resource-constrained hardware necessitate innovative approaches that transcend traditional precision and overparameterization paradigms. My goal is to bridge the gap between the research in the neural network field and custom hardware field, to push the limits of the trade-off between accuracy and computational cost. E. Wang et al. [1] showed that algorithmic advances in deep learning increasingly benefit from specialized hardware, with FPGAs outperforming alternative platforms. Motivated by this, I aim to better understand FPGA architectures-particularly their K-input Boolean Lookup Tables (LUTs)-to co-design algorithms and hardware more effectively. LUTNet [2], introduced by E. Wang et al., was the first to use K-input LUTs as inference operators, enabling aggressive pruning and significant area savings. Constantinides [3] explores the limitations of binary neural networks (BNNs), which use binary values for weights and activations, and presents a critical analysis of why they often fail to achieve the same level of accuracy as networks using other data representations. The reason does not lie within the generality of the data representation, but rather in the traditional design technique that is unable to adapt the topology of the network to the underlying datatype. Hence, moving forward, we need to find a way to adapt the neural network topology to both the data and the nature of the discrete representation of activations.

My initial research question is: What network topologies are best suited for BNNs? To answer the question, I would try to investigate the limitations and find existing/developing new topologies that address these problems. Following, I would explore what modifications would benefit the design, these modifications coming from both machine learning and hardware perspectives.