Green Brain: Computational Modelling of the Honeybee Brain

Lead Research Organisation: University of Sheffield
Department Name: Computer Science

Abstract

Building intelligent machines that can perform complex cognitive tasks as well as or better than the human brain is a long-standing challenge of modern science. This quest has seen one of its highlights when IBM's Deep Blue chess computer beat the world champion Kasparov in 1997. Despite its superior performance in chess, this system was however in no way similar to or as powerful and versatile as a brain. More recently the Blue Brain initiative, also partially funded by IBM, set out to build a real-scale model of a cortical column of the human brain, moving us closer to the goal of eventually building an artificial brain that works like its biological counterpart.

In the Green Brain project we propose to build such an artificial brain, but of the smaller brain of the honeybee. We will work with the world-leading research group of Prof. Martin Giurfa in Toulouse, who are experts in all aspects of bee brain anatomy, physiology and bee cognition and behavior. Bees have a surprisingly large cognitive capacity including transfer of learned associations across sensory modalities, e.g. from smells to colors, and learning abstract concepts such as the categories of "the same" and "different". At the same time their brains are much smaller, structured and (proportionally) much better researched than the complex human brain. It is also much easier to perform invasive manipulations to dissect how different parts of the bee brain function.

In the Green Brain project we will build detailed computer models of the two most important sensory systems of the bee, the senses of smell (olfactory system) and of sight (visual system). In doing so, we will incorporate existing data, models and principles and identify further how they give rise to the observed impressive cognitive abilities. We will then combine the sensory systems with learning models and models of sensory integration in close collaboration with the experts in the Giurfa lab to eventually build a full-scale model of the bee brain. This model will be implemented on state-of-the-art massively parallel graphical processing unit (GPU) based super-computers, a new technology spearheaded by NVIDIA Corporation who is supporting this project with GPU hardware donations. Using GPU computing will allow us to simulate our Green Brain model in real time, which will be essential for the final phase of the project when we will put the Green Brain to work as the brain of an autonomous flying robot. This is an important further advance over current work on brain models because it is becoming more and more clear that an essential aspect of brain function is that the brain is not acting in isolation but in constant interaction with the body and the environment. This concept of "embodiment" and its consequences for cognition are important insights of modern cognitive science and will become equally important for modern neuroscience.

The outputs from the Green Brain project will have impacts in several academic areas. In the neurosciences it will advance the field of large scale brain models and our understanding of how information is processed in the sensory systems of bees. We will also contribute new tools for the use of modern GPU technology for artificial brains and employing them in bio-mimetic robotics. For the cognitive sciences we will contribute to the understanding of embodiment in biologically realistic model systems.

Beyond academia, the development of autonomous flying robots may have applications in environmental exploration, search and rescue and artificial pollination. Developing a better understanding of the mechanisms underlying cognition may ultimately translate into greater insights into human cognition and cognitive disorders. Finally, developing a better understanding of the honeybee may prove to be important in its own right as bees are a key pollinator in most ecologies and hence a 'keystone species' and vital for food security.

Planned Impact

This project combines technology development with substantial basic research components. The ultimate impact of the technologies developed is relatively easy to predict. The impact predicted from basic research is necessarily much more speculative, however it is easy to demonstrate the economic, ecological and societal importance of the basic research problems being tackled. The potential impact of the project can be summarised as follows:

* supporting the development of autonomous and adaptive flying robots, with potential application in environmental exploration, search and rescue, artificial pollination, etc.
* developing greater understanding of the mechanisms underlying cognition, which may ultimately translate into greater understanding of human cognition.
* developing greater understanding of the honeybee, a key pollinator in most ecologies and hence a 'keystone species' and vital for human food security.

These potential impacts are described in more detail below:

Towards development of autonomous and adaptive flying robots
---
This impact is the most immediate to arise from the project. This project will develop sophisticated behaviours for control of flying robots, and demonstrate their embodiment in a testbed flying robot platform. There are obvious and innumerable applications of autonomous, adaptive flying robots. These include exploration in dangerous environments, such as search and rescue. Less obvious applications may include artificial pollination, which might become essential for human food security if current declines in honeybee populations continue. For this reason, the US National Sciences Foundation recently awarded Harvard University and partners one of three 'Expeditions in Computing' awards, worth up to $10m, to fund the Robobees project which is developing flying microrobots. Control of these is a hard problem, and an obvious potential application area for the current project which we shall explore during its development.

Understanding basic mechanisms of cognition
---
The basic research undertaken in the project may have longer term economic and societal impacts. The cognitive sophistication of the honeybee brain is often under-appreciated. In fact, honeybees exhibit many of the cognitive abilities of 'higher' organisms such as vertebrates, including humans. Understanding the mechanisms of cognition in the honeybee may ultimately translate into a greater understanding of animal and human cognitive mechanisms. Since understanding human cognitive mechanisms should help with treatment of cognitive disorders, the societal and economic benefits of such research are potentially very large. For example Parkinson's disease alone, which may be caused by malfunction of basic action selection and learning circuits in the human brain, has been estimated to cost the UK economy between £0.45bn and £3.3bn per year.

Ecological, economic and societal significance of the honeybee
---
Honeybees are vital pollinators, and thus are 'keystone species' whose absence from an ecosystem can have catastrophic consequences for other species. Human food security in particular is heavily reliant on pollination by honeybees. As has been widely reported, however, honeybee populations have been in sharp decline in Europe and North America. For this reason, the UK government made funding of £10m available in 2009 for investigation into the causes of honeybee decline, with similar funding initiatives in the USA. One example motivation of the NSF funded Robobees project described above is to produce 'artificial pollinators' to take over in the event of a complete collapse in honeybee numbers; the current project could ultimately contribute to such a technological solution. More fundamentally, while progress on identifying the causes of honeybee population decline has been made, further basic research into honeybees, including behavioural and neuroscience research, can only improve our understanding of them.

Publications

10 25 50
 
Description We have discovered how optic flow, the visual motion sensed by the, can be estimated in a very robust way by a plausible and simple model of the honeybee visual system. We have shown how this can be used to guide a simulated bee through a virtual environment, such that its behaviour closely mimics that of real bees trained to do the same task. This research is of potential use for robotics applications.
Exploitation Route Our published results could be useful to those developing controllers for autonomous robots.
Sectors Other

URL http://greenbrain.group.shef.ac.uk
 
Description An algorithm developed during the Green Brain project (Cope et al. 2016 PLoS Comp Bio) led to international patent filings including the US (US patent no. 11,170,646) and the formation of a thriving deep tech startup, Opteran (www.opteran.com) which is now going through it's second seed funding round, raising over £10m from US and European investors.
First Year Of Impact 2020
Sector Aerospace, Defence and Marine,Other
Impact Types Economic

 
Description Brains on Board: Neuromorphic Control of Flying Robots
Amount £4,816,675 (GBP)
Funding ID EP/P006094/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 12/2016 
End 12/2021
 
Description Faculty of Engineering Widening Participation Activities
Amount £5,540 (GBP)
Funding ID Virtual Robotics 
Organisation University of Sheffield 
Sector Academic/University
Country United Kingdom
Start 02/2016 
End 06/2016
 
Description Royal Society International Exchanges
Amount £12,000 (GBP)
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 05/2015 
End 04/2017
 
Description Honeybee computational neuroscience collaboration with Andrew Barron 
Organisation Macquarie University
Country Australia 
Sector Academic/University 
PI Contribution Computational modelling expertise
Collaborator Contribution Expertise in invertebrate neuroscience and behaviour
Impact -
Start Year 2014
 
Description Member of the NineML standardisation committee 
Organisation National Center for Scientific Research (Centre National de la Recherche Scientifique CNRS)
Country France 
Sector Academic/University 
PI Contribution NineML is a simulator independent language with the aim of providing an unambiguous description of neuronal network models for efficient model sharing and reusability. As a member of the standardisation committee I am involved in completing the NineML specification and generating code to allow conversion between the NineML and SpineML description formats.
Collaborator Contribution All members of the committee are involved in completing the NineML specification, and various members have responsibility for creating and maintaining software tools for writing and manipulating NineML based models.
Impact The outcomes of this collaboration are the NineML model description format and the updated tools for generating and manipulating NineML based models.The collaboration involves the fields of computational neuroscience and neuroinformatics.
Start Year 2014
 
Description Member of the NineML standardisation committee 
Organisation Okinawa Institute of Science and Technology
Country Japan 
Sector Academic/University 
PI Contribution NineML is a simulator independent language with the aim of providing an unambiguous description of neuronal network models for efficient model sharing and reusability. As a member of the standardisation committee I am involved in completing the NineML specification and generating code to allow conversion between the NineML and SpineML description formats.
Collaborator Contribution All members of the committee are involved in completing the NineML specification, and various members have responsibility for creating and maintaining software tools for writing and manipulating NineML based models.
Impact The outcomes of this collaboration are the NineML model description format and the updated tools for generating and manipulating NineML based models.The collaboration involves the fields of computational neuroscience and neuroinformatics.
Start Year 2014
 
Title MOTION DETECTORS 
Description Examples relate to detectors for determining motion from visual cues, said determining being invariant to changes in at least one of temporal and spatial frequencies of said visual cues. 
IP Reference WO2018203094 
Protection Patent application published
Year Protection Granted 2018
Licensed Yes
Impact This discovery forms part or all of two Journal Publications (https://doi.org/10.1371/journal.pcbi.1004887 & https://doi.org/10.1371/journal.pone.0172325) and is key to ongoing research and commercialisation
 
Title SpineCreator 
Description The SpineCreator graphical tool is a cross platform (OSX with retina support, Linux and (potentially) Windows) graphical drag and drop tool for SpineML model editing and simulation. It features OpenGL visualisation of networks, simulator integration and graphical analysis tools. See http://bimpa.group.shef.ac.uk/SpineML/index.php/Gui for details. 
Type Of Technology Software 
Year Produced 2014 
Open Source License? Yes  
Impact The software is being used in the European seventh framework project NoTremor (http://notremor.eu/notremor/) 
URL https://github.com/SpineML/SpineCreator
 
Title SpineML_2_GeNN 
Description SpineML_2_GeNN converts neural models described in the SpineML description format into the GeNN GPU simulation framework 
Type Of Technology Software 
Year Produced 2014 
Open Source License? Yes  
Impact This software allows computational neuroscience models to be easily simulated on the embarrassingly parallel hardware of modern NVidia graphics cards, vastly decreasing the cost required to improve simulation performance. 
URL https://github.com/genn-team/genn
 
Company Name OPTERAN TECHNOLOGIES LIMITED 
Description Opteran Technologies is pioneering natural intelligence for autonomous systems, as an alternative to data- and compute-intensive machine learning approaches. The company's product portfolio includes solutions for optic-flow based control and navigation, object detection and classification, as well as multi-camera hardware solutions; it provides systems integrators with access to Opteran technology via its hardware development kit. Founded in March 2020, Opteran closed its £2.1m seed round in November 2020 led by IQ Capital, with Join, Episode 1, Seraphim, and angel investors. 
Year Established 2020 
Impact Opteran is in the process of signing up commercial customers under development licences.
Website https://www.opteran.com/
 
Description Aerospace America IS Year-in-Review 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Green Brain Project was highlighted in the Intelligent Systems Year-in-Review in the annual Aerospace America magazine written by AIAA.
Year(s) Of Engagement Activity 2015
URL http://www.aerospaceamerica.org/
 
Description BBC Click television broadcast coverage 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Television broadcast coverage of Green Brain project, and interviews with PI (James Marshall) and research associate (Alex Cope) on BBC Click
Year(s) Of Engagement Activity 2015
URL http://www.bbc.co.uk/programmes/p02npxlq
 
Description Bee-Hind a Robot Mind Activity 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Day long workshop for school children to activity engage in Robotics, Artificial Intelligence, & Neuroscience including hands-on activities. Included 30 students which attended after a previous Public Engagement talk at their school (a Widening Participation school). Students were engaged and interest.
Year(s) Of Engagement Activity 2015
 
Description Bee-ing Sherlock Holmes Outreach Activity 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact Activity engaged students, encouraged discussion, and sparked questions related to modelling the brain.

Students gave very positive feedback about the activity and enjoyed the introduction into modelling. This activity also led to future Excellence Hubs workshops led by researchers in the Kroto Research Institute.
Year(s) Of Engagement Activity 2014
URL http://www.york.ac.uk/excellencehub/subject-taster-events/eventsarchive/archive-13-14/subject-taster...
 
Description Bee-ing Sherlock Holmes Outreach Activity 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Schools
Results and Impact Activity engaged students, encouraged discussion, and sparked questions related to modelling the brain.

Students gave very positive feedback about the activity and enjoyed the introduction into modelling/engineering. Had one student in particular talk to me about doing work experience in our lab for the coming summer.
Year(s) Of Engagement Activity 2014
URL https://www.education.gov.uk/futurescholarawards/Venues/Venue.aspx?VID=445&EVID=2346
 
Description Beekeepers Talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Talk to Sheffield Beekeeper's Association about what we have learned about bees in the Green Brain Project
Year(s) Of Engagement Activity 2015
 
Description Digital Human BBC Radio 4 broadcast interview 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed as part of Digital Human broadcast (BBC Radio 4)
Year(s) Of Engagement Activity 2015
URL http://www.bbc.co.uk/programmes/b05rnyz2
 
Description Discovery Night 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Booth at the Discovery Night exhibition at the University of Sheffield.
Year(s) Of Engagement Activity 2016
URL http://www.sheffield.ac.uk/discoverynight/programme/robots
 
Description Drones Save Bees 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article on use of drones in the Green Brain to show drone uses in conservation and agriculture to help increase public view of them.
Year(s) Of Engagement Activity 2016
URL http://www.dronesfutureskies.com/environmental-impact#drones-save-bees
 
Description Headstart Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact HEADSTART
"Bee-hind a Robot Mind"
Tue, July 12, 2016, 1:00pm - 5:30pm
The Diamond, University of Sheffield
Year(s) Of Engagement Activity 2016
 
Description IUSSI 2014 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Talk sparked questions and discussion afterwards

After my talk there was interest expressed in the work.
Year(s) Of Engagement Activity 2014
 
Description Invited Robotics Mini-Lecture 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Robotics Mini-Lecture
Wed, March 30, 2016, 6pm - 8pm
Retford Post 16 Centre, Old Hall Dr, Retford DN22 7EA, United Kingdom
Year(s) Of Engagement Activity 2016
 
Description Invited Talk - Drone Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited Oral Presentation
Drone, Data X Conference
"Green Brain Project"
Mayo, Ireland
November 2015.
Year(s) Of Engagement Activity 2015
 
Description Invited conference talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk at the Neurosciences R&D Conference, Barcelona, December 4-5th 2015
Year(s) Of Engagement Activity 2015
 
Description Invited talk at workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk at the 2nd Workshop on Code Generation at the Forschungzentrum in Julich, Germany
Year(s) Of Engagement Activity 2016
 
Description Media coverage including New Scientist and Sunday Times 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact -

Coverage included (among others)

"Summon the bee bots: Can flying robots save our crops?", November 18th November 2013, New Scientist (on the cover).
"Flight of the robo-bee to save fruit crops", 7th October 2012, The Sunday Times.
Year(s) Of Engagement Activity 2012,2013
 
Description Mobile Uni Public Engagement Talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Talk sparked questions and discussion afterwards with several members of the general public.

After my talk, a student from the Media Studies department approached me and did a science journalism piece on the project.
Year(s) Of Engagement Activity 2013
 
Description National Science and Engineering Week Talks 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Regional
Primary Audience Schools
Results and Impact Activity engaged students, encouraged discussion, and sparked questions related to robotics.

Students were engaged, gave very positive feedback about the activity, and enjoyed the introduction into robotics.
Year(s) Of Engagement Activity 2014
 
Description Oral presentation in Bernstein Conference, Green Brain workshop in Goettingen (2-5 September 2014) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The talk helped better understanding of our goal and achievements by colleagues and it was followed by a discussion.

After the talk I could discuss with experts in my field about my and their methods and results.
Year(s) Of Engagement Activity 2014
URL http://www.nncn.de/en/bernstein-conference/2014/workshops/green-brain-how-models-of-invertebrate-bra...
 
Description Programming a Robotic Bee Outreach Activity 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Regional
Primary Audience Schools
Results and Impact Activity engaged students, encouraged discussion, and sparked questions related to robotics.

Students gave very positive feedback about the activity and enjoyed the introduction into programming. This activity also led to future Excellence Hubs workshops led by researchers in the Kroto Research Institute.
Year(s) Of Engagement Activity 2014
URL http://www.sheffield.ac.uk/kroto/news/outreachday
 
Description Robot Mind Programming Activity 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Repeat of the Bee-Hind a Robot Mind programming activity used in past outreach events. This activity was requested by the University for this program after the demonstration of success from earlier implementations. One student even reported that it was "the best one yet because it felt like we were activity doing something!" in feedback.
Year(s) Of Engagement Activity 2016
 
Description Science Week Talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Public Engagement talk to school children in Widening Participation school. Gave intro to the Green Brain Project, Robotics, and AI. Follow-up interactions with this school included hosting them for a day-long workshop at the University of Sheffield to more activity participate in research topics.
Year(s) Of Engagement Activity 2015
 
Description Sheffield Science Exhibition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Booth at the Sheffield Science Exhibition which reached an audience of ~500 Students of Ages 5-10.
Year(s) Of Engagement Activity 2015
 
Description Virtual Robotics Outreach Project 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Faculty of Engineering Widening Participation Activities (February - June 2016)
Title: Virtual Robotics
Aimed to support young people from a disadvantaged community to engage in cutting-edge research, as well as accessing university ideas and raising aspirations
Grant Award Amount: £5,540 Total
Year(s) Of Engagement Activity 2016
 
Description Work Experience Student 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Hosted Work Experience student from a local Rotherham school. The student developed an algorithm for a robot to do a waggle-dance in a week with us and is pursuing university in robotics.
Year(s) Of Engagement Activity 2016
URL https://www.youtube.com/watch?v=BTVB17ps_vc