Comparing biological and computer vision: an AI approach to visual guidance in Harris' hawks.

Deep learning techniques are becoming increasingly popular in the design of flight controllers of micro-aerial vehicles (MAVs). They equip these vehicles with awareness of their environment, and allow them to take intelligent navigational decisions. Though impressive developments have been achieved, these advances still lag behind birds' performance, especially in terms of manoeuvrability and integration in urban or cluttered environments. In order to further expand the potential of MAVs, we need them to avoid obstacles and follow goals with computationally efficient and sensor-efficient methods, that don't restrict the vehicle's limited payload capacity. RGB cameras have proven to be very informative sensors for navigation. Birds, and raptors in particular, strongly rely on vision for carrying out target tracking and obstacle avoidance tasks in flight. Can we learn from them new vision-based navigational strategies to solve these guidance problems? To answer this question, we propose a framework based on motion-capture experiments with trained Harris' hawks, and a deep learning model that captures the bird's strategy, considering exclusively visual input. This approach provides several advantages: first, deep learning models are well-suited to capture the complex dynamics of such behaviours in an end-to-end fashion (from raw sensory inputs to high-level control commands); second, by building a bird-mimicking network we can evaluate which encodings of the visual information (texture, optic flow, contrast changes, etc.) are the most relevant for a successful imitation of the bird behaviour; third, it allows us to easily compare with other computer vision solutions and guidance approaches currently used in MAVs for obstacle avoidance and target tracking. Finally, we consider an application of the bird-imitating strategy to MAV guidance in simulation.

The proposed project addresses several of the BBSRC priority areas:
animal behaviour, use of mathematical tools for biology, STEM approaches to biology, systems approaches to the biosciences, technology development for the biosciences and data driven biology.