"Common Sense" and flexible learning in AI agents: Do current AI agents possess the "basic skills" necessary for them to enter the workforce?

Policy and institutional frameworks for AI governance rely on up-to-date information about AI capabilities. It is now possible to train AIs to exceed human performance on numerous, individual tasks such as classifying images or analysing large datasets. However, these systems cannot act outside of the task they have been trained for and often fail under even minor deviations from their expected inputs (Shevlin et al., 2019). Basic cognitive skills such as object permanence are central to flexible function, acquired in human infanthood, but are a major challenge for AI (e.g. Voudouris et al., 2022) and their development would represent a step-change in potential applications. However, current AI benchmarks are neither sufficiently cognitively defined nor sufficiently general to measure performance in these skills.

This studentship will form part of a project taking a new approach to AI evaluation - inspired by cognitive science both in terms of concepts (which is already common across AI research) and in applying a conceptual and methodological framework that is comprehensive and robust. Jointly supervised by Dr Lucy Cheke (Cognition and Motivated Behaviour Lab, Psychology; Director of the kinds of Intelligence Program, Leverhulme Centre for the Future of Intelligence), who has led research in developmental/comparative psychology and AI evaluation, and Dr Flavia Mancini (Computational and Biological Learning
research group, Engineering), who leads an interdisciplinary research group in computational neuroscience and AI. The student will create and train novel artificial agents using techniques in Deep Reinforcement and Bayesian learning while in parallel developing a series of cognitive tasks within the Animal AI platform (http://animalai.org) to assess the capabilities of these agents. They will benchmark this performance against that of children. Finally, together with both supervisors, they will learn how to computationally model behavioral data, using both Bayesian and RL approaches, to extract a nuanced and
comprehensive understanding of the "cognitive fingerprint" across different tasks, for both children and agents generated using different architectures.