PhD in Computer Science - Video Object Segmentation (VOS) In Egocentric Video

The main research focus on my PhD is studying the video object segmentation in egocentric videos where someone is holding a camera on the head, the challenge is being able to identify the surrounding objects as well as tracking each object thought the video to be able to recognize the actions and the relationship between objects which would be very helpful for many applications such as assistive robotics as the robot would understand the environment and thus give better assistance to the user. An important part of the project is to create a new dataset from EPIC-KITCHENS [1] dataset called VISOR, we created manual segmentations for objects in kitchens to: (1) be able to study the current state-of-art architecture on the egocentric setup and; (2) refine the current architectures and work on new ones that could tackle the egocentric problems such as the motion blur and the fast motion (3) publish the first egocentric segmentation-based dataset which expected to help community understanding the challenges in new environments (kitchens) since most of the current datasets are from YouTube or outdoor environment
We're working on the project with the collaboration of researchers from (1) University of Michigan (US) where they have experience on hand-object interaction in the cases when we're intended to understand the actions from the hand and objects masks (2) University of Toronto (Canada) who have tools for the segmentation as well as experience in the image level segmentation. We're aiming to merge this experience toward egocentric video understating. This project falls within the EPSRC Image and vision computing research area.

[1] Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., ... & Wray, M. (2018). Scaling egocentric vision: The epic-kitchens dataset. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 720-736).