Performing Activity Recognition on Unscripted Culinary Activities in a Living Lab

Culinary activities, including the preparation and consumption of sustenance, have the
potential to provide valuable insights into the mental and physical health of individuals.
Behavioural habits or variances which may become evident through monitoring of these
processes can be symptomatic of conditions such as diabetes, obesity, depression and
dementia. Therefore, autonomous monitoring and understanding of activities that take place
in the kitchen is an important goal that if achieved could improve the lives of many people in
the comfort of their own homes.
A key part of autonomous home monitoring systems is activity recognition, as this allows for
machine understanding of human situations and actions. This project focuses on
knowledge-based activity recognition, since this has more flexibility for smaller data sets.
The sequences used to create and test our model will be recorded in the Bristol SPHERE
house, which already has an assortment of cameras and sensor installed. Sequences will
be unscripted, allowing for a wide range of interesting and challenging behaviours.
A Computational State Space Model will form the foundation of the recognition system, with
marginal filtering used to determine the most likely state at each moment in time. During the
course of the project, elements of this will be enhanced or upgraded in order to improve
model performance. Action recognition will be improved through the integration of visual
data through Convolutional Neural Networks, which will be compared to current feature
based approaches in terms of speed and accuracy of classification. In addition, recognition
will be improved to ensure that classification is invariant to different locations and system
configurations. Activity recognition will be upgraded to allow the model to process activities
performed by multiple individuals in the same environments. Additional improvements will
be made to improve efficiency by reducing the overall state space through simple logical
changes. Data permitting, data-driven approaches to activity recognition will also be trialled
and compared to previous methods.
The implications of this project are wider reaching than it's use in the kitchen. Applications to
other environments or situations could have a wide range of different uses in many different
sectors. Mainly, this project also builds upon and enhances the work being done by the
SPHERE project.