Exploiting invisible cues for robot navigation in complex natural environments

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics

Abstract

Outdoor navigation in natural environments remains a challenge for robotics. Recent breakthroughs in robot navigation have been dependent on specific sensor technologies, such as laser depth sensors and GPS, and advanced image processing. The ability of animals such as ants to navigate effectively without such power- and computation- hungry systems are a proof of principle that alternative cheaper approaches are viable. Ants also have specialised sensing, with a peripheral visual system that has evolved to be sensitive to crucial cues for navigation. Specifically, they make use of non-visible (to humans) light cues in the form of ultraviolet (UV) and polarised light detection. UV detection allows the important signal of the horizon shape against the sky to be easily distinguished. Polarised light detection provides an external compass cue of heading relative to the sun direction, even when only a small portion of the sky is visible.
We propose to build a sensory system that gathers the full range of light cues available to the ant, in its natural ecological situation, and to analyse the information contained in this signal. We will also analyse how the specific sensor layout (ommatidia array), peripheral receptor characteristics, and the motor behaviour of the ant may contribute to extracting salient information. The data will form a test-bed for comparison of algorithmic and neural models of the processing that underlies the navigation capabilities of the ant. To date, these cues have been considered separately but we believe the navigational success of this system depends on the specific combination. For example, the directional information in the polarised sky may form an important part of visual memories; and UV information may contribute to disambiguation of the polarisation pattern and the robustness of this information under different cloud conditions.
There has been a substantial increase in the last few years in research into insect neural pathways involved in processing these cues which has yet to be exploited in robot models. In particular there has been breakthrough work on the central brain mechanisms involved in decoding polarised light to obtain heading direction. There is also a rapidly increasing understanding of the circuits involved in learning, a key component of navigation capabilities.
A key aspect of our method, in comparison to many bio-inspired projects, is that we do not propose to start by designing and manufacturing a sensor as similar as possible to the ant eye. Rather, we consider it essential to first understand what aspects of the eye are actually key to support navigation, and what variation from the exact biological design will conserve the desirable properties while simplifying the manufacture. The final outcome of this research will be a detailed design for a small scale and low power vision system for outdoor navigation. We hypothesis that we can leverage the miniaturisation of CCD cameras (which are normally sensitive to the upper range of UV, but have filters applied to remove it) to compose an omnidirectional compound eye where the filter characteristics of each ommatidia are appropriately tuned for wavelength or polarisation plane, in imitation of the ant eye. Beyond the direct application to robot navigation in natural environments, in tasks such as agriculture, search and rescue, and environmental clean-up, such a design has potential to be developed for other applications, including mobile devices, inspection and surveillance.

Planned Impact

UK and European governments have highlighted the robot industry as a key research and growth sector. Currently the market is still dominated by industrial robots confined to factories. Robotic cars have recently shown that mobile autonomy outdoors is possible yet are reliant on energy and computationally expensive sensing (e.g. 3D laser scanners). Development of a novel low-cost sensor for robot navigation is immediately applicable to this emerging market. Mid-term applications for the system include self-guided systems; robots in agriculture; exploration; environmental monitoring and clean-up operations. We also predict there will be potential applications other than robotics for miniaturised, omnidirectional UV and polarisation sensing.

The outcome of this proposal will be a prototype system with a design for manufacture, and patenting possibilities. We will fully investigate the options for commercialisation either through a university spin out or a licencing agreement with industrial partners. Primary market research shall be conducted by attending targeted industrial events at which companies likely to be interested in autonomous navigation (e.g. specialist robot developers, automotive manufacturers, defence contractors) are in attendance. Looking beyond the specific device we propose to develop here, we also hope to demonstrate to industry the real benefit of exploiting natural solutions for specialised sensing problems, to increase direct investment in future research in biomimetics through industrial-academic partnerships. The work undertaken here will provide training and experience in commercialisation to the junior project participants and may ultimately contribute to continued expansion and employment opportunities in this area.

The key industries where we expect impact are areas of social importance and the approach can contribute in the long term to environmental sustainability, safety and energy reduction goals. In contrast to the conventional public image of robotics as aiming for human-like machines, the research is premised on building small cheap devices that may make a big difference. We look to involve the public and stakeholders in this emerging technology through a series of science communication activities, as detailed in the Pathways to Impact.
 
Description We have developed a model of processing of polarised light in the insect eye and brain, and shown how this can effectively extract compass information from the pattern in skylight, which forms the basis of a proposed new design for a robot sky compass sensor. The output has been incorporated into neural algorithms, inspired by the ant brain, that can explain insect navigation behaviour and can be used on robots. The models have been tested in a realistic reconstruction of the ant's natural environment, and compared to results from tracking ants in natural terrain. This was supported by the development of general software tools allowing meshing of natural scenes from laser scan data, and tracking of animals using a handheld camera under natural conditions. We have also shown how the use of ultraviolet light in conjunction with a spherical harmonic encoding method can improve robot localisation based on sky segmentation under challenging conditions. We have carried out new behavioural experiments showing that ants are capable of navigating backwards and can transfer information between their terrestrial visual memory and celestial compass.
Exploitation Route The methods developed should be applicable to navigation technology, particularly for low-cost systems in natural contexts, such as environmental monitoring and agriculture. The neural models we have developed are also of general interest to understand cognition in insects.
Sectors Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Electronics,Environment,Transport

 
Description Some findings from this research are now being translated to use on robot platforms in industry settings.
First Year Of Impact 2021
Sector Aerospace, Defence and Marine
Impact Types Economic

 
Description AHDB PhD Studentship
Amount £70,388 (GBP)
Organisation Agricultural and Horticulture Development Board 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2017 
End 09/2020
 
Description Discovery Projects
Amount $483,000 (AUD)
Funding ID DP160102658 
Organisation Australian Research Council 
Sector Public
Country Australia
Start 01/2016 
End 12/2018
 
Description EPSRC Impact Acceleration Account
Amount £36,898 (GBP)
Funding ID PIV068 
Organisation University of Edinburgh 
Sector Academic/University
Country United Kingdom
Start 06/2021 
End 03/2022
 
Description Responsive mode
Amount £286,057 (GBP)
Funding ID BB/R005052/1 
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 01/2018 
End 12/2020
 
Description SkyEye: Feasibility Study of Vision Based Localisation in GPS-compromised Environments
Amount £24,993 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2022 
End 03/2022
 
Description UltimateCOMPASS ERC consolidator grant
Amount € 188,924 (EUR)
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 06/2019 
End 06/2024
 
Title Habitat3D 
Description Habtat3D is an open source pipeline that converts multiple point counts into a mesh allowing visualisation, adaptioation, and analysis 
Type Of Material Improvements to research infrastructure 
Year Produced 2018 
Provided To Others? Yes  
Impact We have had contact from researchers within and outside of our field looking to use this tool for their own research. 
URL http://www.insectvision.org/3d-reconstruction-tools/habitat3d
 
Title Habitracks 
Description Habitracks is a software tool for researchers looking to track animals in their natural habitats. 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact This tool has already been adopted by a number of international laboratories for use in their behavioural experiments. 
URL http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w41/Risse_Visual_Tracking_of_ICCV_20...
 
Title roboant 
Description Robot platform for algorithm prototyping 
Type Of Material Technology assay or reagent 
Year Produced 2017 
Provided To Others? Yes  
Impact This platform has been used by a number of students for various projects. 
URL https://blog.inf.ed.ac.uk/insectrobotics/roboant/
 
Title 3D Mesh of ant habitat in Canberra, Australia 
Description Reconstruction of an ant habitat (Myrmecia croslandi) based on 9 clouds captured in Canberra, Australia. The 8800sqm area features several complex Eucalyptus trees. This reconstruction was tuned to demonstrate compressibility features of Habitat3D: While strongly compressing the ground (>99% size reduction; file: ground.ply) all characteristics of the complex trees are preserved (file: vegetation.ply). 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact This environment forms a cornerstone of attempts to assess how insects visually navigate their environments. Combined with additional tools research groups will be able to benchmark their models using this data. 
URL http://www.insectvision.org/3d-reconstruction-tools/habitat3d
 
Title 3D mesh of ant habitat in Seville, Spain 
Description Reconstruction of a natural foraging environment of desert ants (Cataglyphis velox) based on 56 laser scans and covering an 1018sqm area. This highly cluttered terrain features hundreds of individual plants and the reconstruction was tuned towards overall accuracy: Both, the ground and vegetation reconstruction feature millimetre accuracy while reducing the memory load to 1.5%. The reconstruction is divided into vegetation (inner_vegetation.ply and outer_vegetation.ply) and ground (inner_ground.ply and outer_ground.ply). 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact This environment forms a cornerstone of attempts to assess how insects visually navigate their environments. Combined with additional tools research groups will be able to benchmark their models using this data. 
URL http://www.insectvision.org/3d-reconstruction-tools/habitat3d
 
Title Wildlife Animal Tracking (WAT) dataset 
Description A dataset of small animals tracked in their natural habitats 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? Yes  
Impact This dataset presents a new benchmark for new tracking software 
URL http://blog.inf.ed.ac.uk/insectrobotics/WAT
 
Description Lund 
Organisation Lund University
Country Sweden 
Sector Academic/University 
PI Contribution Computational modelling of path integration in the central complex of the bee.
Collaborator Contribution Neurophysiological and neuroanatomical data on the central complex
Impact Paper in Current Biology (multidisciplinary: biology, computing, robotics) Two proposals (in submission/preparation) for funding from the Swedish government to explore the transfer of ideas from insect neural circuits to optical computing with nano components (multidisciplinary: biology, computing, physics, engineering)
Start Year 2015
 
Description Lund2 
Organisation Lund University
Country Sweden 
Sector Academic/University 
PI Contribution I have supervised several projects that implement the behaviour of the Dung beetle on a robot using a neural model we devised.
Collaborator Contribution The partner at Lund, Marie Dacke, has contributed data and advice to the student projects.
Impact A grant from ERC to support further work has been awarded. This is a multidisciplinary collaboration, combining behavioural biology, neuroscience, modelling and robotics.
Start Year 2016
 
Description Modelling the honeybee polarization compass 
Organisation University of Wurzburg
Country Germany 
Sector Academic/University 
PI Contribution Providing the computational model and collaborating on its adaptation to data from the honeybee
Collaborator Contribution Providing data on the honeybee
Impact The partner at Wurzburg was awarded a DFG grant to support this work. The collaboration was specifically described as part of the successful proposal.
Start Year 2020
 
Description Sussex 
Organisation University of Sussex
Country United Kingdom 
Sector Academic/University 
PI Contribution Joint work on behavioural experiments on ants which is now linked to model development and a funded grant proposal
Collaborator Contribution Behavioural and neural methods for ant experiments
Impact Multidisciplinary - robotics and biology
Start Year 2016
 
Description Toulouse 
Organisation University of Toulouse
Country France 
Sector Academic/University 
PI Contribution Joint work on behavioural experiments in desert ants
Collaborator Contribution Joint work on behavioural experiments in desert ants
Impact There have been a number of journal articles, posters, and talks directly resulting from this collaboration. All are listed in the appropriate sections.
Start Year 2016
 
Title Habitat3D 
Description Habitat3D is an open source cross-platform tool to generate photorealistic meshes from point clouds of natural outdoor scenes. All necessary processing steps (filtering, segmentation, feature extraction, meshing, etc.) as well as recipe-based generation of pipelines are incorporated in a GUI-based framework. Habitat3D is written in C++ and requires Qt, PCL, VTK and BOOST. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact The tool has been used to recreate ant environments from our own and others field studies 
URL http://www.insectvision.org/3d-reconstruction-tools/habitat3d
 
Title Habitracks 
Description Habitracks provides a tool for tracking small objects in images taken with a moving camera 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact This software is already being adopted by other researchers in the field 
URL http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w41/Risse_Visual_Tracking_of_ICCV_20...
 
Description Backwards ants press coverage 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact A press release was made describing the results of our Current Biology paper "How Ants Use Vision When Homing Backward". The story was picked up internationally and appeared in the following venues: DAILY MAIL, DAILY MIRROR, THE HERALD, WESTERN DAILY PRESS, WIRED.CO.UK, AFP, EUROPE 1, L'EXPRESS.FR, FRANCE INFO, RTL TVI, LIBERATION, LE POINT (France) ATS, LE JOURNAL DU JURA (Switzerland) APS (Algeria) LA NOUVELLE GAZETTE, LA MEUSE, NORD ÉCLAIR, VRTNIEUWS.NET (Belgium) MONTEVIDEO PORTAL (Uruguay) DIARIO UNO (Argentina) EL FINANCIERO (Mexico) DIARIO MONTANES, LA VANGUARDIA, LA VOZ DIGITAL, ABC.ES, EUROPA PRESS, EL ESPECTADOR (Spain) IRISH DAILY MAIL (Ireland) PEOPLE'S DAILY ONLINE, PUEBLO EN LINEA, CHINA POST, CHINA TIMES, CAN, AFP, WORKER CHINA, KAN KAN NEWS (China) BERLINGSKE (Denmark) CHRISTIAN SCIENCE MONITOR, TERRA DAILY, DISCOVERY CHANNEL SCIENCE NEWS, TECH TIMES (USA) EXPATICA (Netherlands) ENCA, ARGUS, NEWS24, (S Africa) NEW VISION (Uganda) THE JAPAN TIMES (Japan) IRAN DAILY (Iran) PAGINA SIETE (Bolivia) WEBINDIA123.COM, INDO ASIAN NEWS SERVICE, SIASAT DAILY, NETINDIA123.COM, DNA INDIA, BUSINESS STANDARD (India) STRAITS TIMES (Singapore) YAM NEWS, EPOCH TIMES (Taiwan) CANADIAN BROADCASTING CORP (Canada) ZME SCIENCE (Romania).
Year(s) Of Engagement Activity 2017
 
Description Presentation at New Scientist Live 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact New Scientist Live is an award-winning festival of ideas and discoveries for everyone curious about science and why it matters, held over five days in London.
Year(s) Of Engagement Activity 2017
 
Description Presentation at VIIHM "Vision for movement" workshop (Univ of Sussex) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact A workshop intended to bring researchers with a common interest in the broad area of vision for movement together to describe methods specific to their fields which could be relevant to other areas of research. Outcomes included new collaborations, shared knowledge, and new funding avenues.
Year(s) Of Engagement Activity 2015
 
Description Public lecture at Macquarie University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact This was a flagship public lecture as part of a series of four organised by Faculty of Science and Engineering at Macquarie University (Australia) with an audience of over 100 people.
Year(s) Of Engagement Activity 2017
URL http://www.mq.edu.au/about/events/view/public-lecture-navigation-in-insects-and-robots/
 
Description Virtual Insect Navigation Workshop Aug 4th-6th 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation at the Virtual Insect Navigation Workshop Aug 4th-6th
Year(s) Of Engagement Activity 2020
 
Description Virtual ant experience 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Using a Oculus Rift headset, we produced a virtual reality game in which the participant experienced the ants point of view in attempting to navigate through the real environment of the ant (a virtual world created from a data set of laser scans in the ant's habitat). This was used at open days and local science festivals.
Year(s) Of Engagement Activity 2016