The computational basis of foraging

You are at a party talking to a work colleague, who is nice but perhaps a little boring. There are other people you could be talking to, but how do you decide when to leave that conversation and move around the room to find someone else? Although a benign everyday example, the decision of whether to stay engaged in a current location or travel to find rewards elsewhere is a fundamental problem that all foraging animals, including humans, have to solve to collect food, drink, materials, and other resources, and seek mates, nesting grounds, places to sleep, and other essentials in their environment. Computational cognitive neuroscience has made great strides in helping us understand how choices are made between two or more fixed options, and the biological mechanisms underlying those choices. However, far less is known about foraging decisions: whether to stay or leave.

There is mounting evidence that foraging choices differ across species, change across the lifespan, and that people with poor mental health differ in when they choose to stay or leave. However, we do not yet understand the computations in the brain that underlie these stay-or-leave choices, and this is holding back neuroscience research aimed at understanding the biology of these choices and why they differ between people. Our aim is to make a major advance in understanding the algorithms governing these choices, to unlock the potential for understanding how foraging choices are made in the brain.

We will provide a new understanding of the computations underlying foraging decisions and provide tools that will let researchers infer these computations from the behavioural data they collect from any species. To do this, we will develop new brain-inspired algorithms for learning and making stay-or-leave decisions. These will draw on a wide range of AI algorithms that have successfully given insight into the processes in the brain in other decision-making problems. We will test if these new algorithms can replicate the well-established behaviours of foragers, like foraging for longer in a location with lots of reward. Then we will examine which of these models can best explain the detailed trial-by-trial decisions observed in existing datasets of foraging tasks tackled by humans, non-human primates, and rodents.

By doing so, we can test whether different brain computations are used in different types of foraging tasks or in different species. Further, we can test hypotheses for why foragers often make apparently non-optimal choices, such as a tendency to stay too long in locations that have become depleted of reward.

To maximise the impact of our work, we will form a foraging interest group, who will meet at strategic points of the grant to offer input and for us to share the knowledge we have gained. Several researchers across different career stages, fields (ecology, psychology, psychiatry and neuroscience), and model species (rodents, primates, and insects), as well as an industrial partner (Opteran Limited) have agreed to take part. Alongside this, the code base for all models will be made freely available, with instructions of how to use them on different types of foraging task.

This grant will make major scientific advances in helping us understand why animals and humans make stay-leave decisions in the way that they do. Moreover, it will offer a range of tools and theoretical platform that can be used by ecologists, psychologists, neuroscientists and psychiatric researchers to understand how the brain makes decisions across species, both in health and disease.

Stay-or-leave decisions are an everyday occurrence for humans and other animals, from committing to watch a new TV series, to seeking a new anthill. Yet the psychology and neuroscience of decision making is largely based on forced-choice tasks between two or more options. Fuller understanding will come by adopting foraging paradigms that test how subjects make stay-or-leave decisions based on rewarding feedback. But we lack neural computational models of how we execute, or learn to successfully make, stay-or-leave decisions. Consequently, the field lacks computational methods necessary to test hypotheses about latent computations, quantify their influence by fitting models to behavioural data in foraging tasks, and make predictions about the neural substrates of stay-or-leave decisions. Our project will tackle these issues by:

1/ Creating and validating a range of neural algorithms for foraging, providing computational tools usable in both basic and applied research and across species. We will build a systematic database of computational models, leveraging new forms of reinforcement learning and evidence accumulation models, and make them freely available for use.

2/ Testing hypotheses of neural computation in foraging, by fitting our algorithms to existing data on human and animal behaviour across different foraging paradigms. We will seek to test hypotheses for why foraging decisions consistently depart from optimal across tasks and species.

3/ Developing a multidisciplinary foraging interest group, to build capacity for cross-disciplinary research at the interface of neuroscience, psychology, biology, and data science.

Our work will enable study of the computational basis of foraging decisions, allowing tracking of changes across the lifespan, in health and disease, and comparisons between species. It could also inform the development of AI agents operating in foraging-like scenarios, from website click-throughs to consumer-choice modelling.