FreeML: Engineering Networked Machine Learning via Meta-Free Energy Minimisation

Inspired by neuroscience, informed by information-theoretic principles, and motivated by modern wireless systems architectures integrating artificial intelligence (AI) and communications, this Fellowship sets out to develop a paradigm-shifting framework for networked machine learning (ML) that is centred on the following ideas.

1. Free energy minimisation: According to the free energy principle, agents optimise internal models so as to minimise their information-theoretic surprise vis-a-vis the available data and prior information. This principle offers a basis to reason about epistemic uncertainty ("know when you don't know") in AI agents that is grounded in information-theoretic analyses of out-of-sample generalisation - away from the current narrow focus on point-wise accuracy, towards uncertainty quantification and calibration. A well-calibrated agent can make informed decisions about when to refrain from acting, about when and how to collect or request more data from the environment or other agents, and about how to guard against anomalies or malicious agents.

2. Networked meta-learning: In meta-learning, agents do not share an ML model in full as in conventional, centralised, solutions. Rather, only a meta-model is shared as a means to transfer knowledge across agents, while enabling the optimisation of personalised local models. As advocated by FreeML, meta-models can naturally implement the engineering principle of modularity by encompassing a common repository of functions that can be combined to suit the cognitive needs of each agent. This framework bridges the gap between the dominant centralised or joint learning approaches - including also federated learning - and the individual learning baseline, by means of limited model sharing, while still enabling meaningful cooperation with a controlled privacy loss.

3. Native integration of wireless communication and learning: Conventional wireless systems are based on the principle of separation between computing and communications. In contrast, the native integration of communications and learning advocated by FreeML embeds wireless communication primitives
as part of the data generating and processing model. Like state-of-the-art integrated solutions, the proposed approach aims at fully utilizing radio channel capacity by avoiding inefficiencies due to separate processing. Unlike existing methods, however, the FreeML framework moves away from the standard problem of communicating under uncertainty (on the communication channel) to the novel problem of communicating uncertainty (on the
solution of the cognitive task) under uncertainty (on the communication channel) in order to support networked meta-learning.

Overall, FreeML sets out to study a novel, theoretically principled, paradigm for ML that moves away from the current centralised, accuracy-focused, state of the art in ML to embrace decentralization via wireless connectivity, uncertainty quantification, personalisation, modularity, privacy preservation, and the right to erasure.

FreeML will involve three industrial partners -- Intel, InterDigital, and Samsung AI -- that will provide guidance and feedback on aspects related to implementation efficiency, communications, and integration with wireless networks, respectively.

This Fellowship proposal builds on the PI's unique inter-disciplinary expertise in information theory, ML, and communications, and is intended to enable a step change in the applicant's career towards a leadership position at the intersection of the fields of engineering and ML/AI. Through this programme, the PI will reach out to a diverse community of STEM students, public, regulators, journalists, and academic colleagues across the two fields to advocate for the central role of engineering for reliable and sustainable ML/AI.