Tracking Whales through Big Data

The focus of this proposed PhD will be to use deep learning to identify the presence of a whale and then to track it as it moves. Two proposed approaches are seen for tracking the whales. The first uses the change in pixel position of the whale in the camera image as it moves to identify trajectories. Initially to develop methodologies to first recognise the whale in a video frame and then identify the relative location compared to the camera using standard maths routines. As the whale moves, it's location in the image will shift and so its new position can be determined and eventually over a number of video frames the whale trajectory can be calculated. The second approach will integrate the ideas of Deep Learning and Reinforcement Learning. In this case, the video stream will be consumed by a Deep Learning agent, which will learn over time how to steer the ROV based on a reward -of keeping the whale in its sights. To begin with, both approaches will use simulated images (generated from a game engine) and library footage. Once developed, fieldwork will commence perhaps starting with tracking simple slow moving creatures such as jellyfish before progressing to whales once the technology is advanced. The PhD will thus be highly applied and involve messy real world images alongside simulations from game engines. This combination is very unusual and the project is likely to produce high quality publications in both deep learning and more general science -projects of this type can, if successful, produce papers in high reputation journals such as Nature or lead to significant press coverage.