Building an End-User Programming Framework for the Internet-of-Things

Lead Research Organisation: The University of Manchester
Department Name: Computer Science

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/S513842/1 30/09/2018 29/09/2024
2109081 Studentship EP/S513842/1 30/09/2018 31/12/2022 Filippos Ventirozos
 
Title Instructional Text Semantic Parsing for Ambient Intelligence 
Description In this study, we propose an End-User programming framework that can aid people who cook. The framework input is the text from cooking recipes which then is used to program IoT devices in the kitchen. The purpose of the current questionnaire-interview is to explore whether and to what extent this automation is helpful for cooks. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact These are five series of interviews with people who cook raging from beginner to experienced challenger level. The questions of the interview aimed to uncover which processes or things during the cooking process impedes or annoys the user. And how technological innovation in the form of ambient intelligence could aid. This qualitative dataset is useful for researchers to focus the technological development for the kitchen in the right direction. 
URL https://figshare.com/articles/dataset/Instructional_Text_Semantic_Parsing_for_Ambient_Intelligence/1...
 
Title Interactive Clustering of Cooking Recipe Instructions:Towards the Automatic Detection of Events Involving Kitchen Devices 
Description Abstract: Cooking recipes are a rich source of semantic information. They contain instructions for food preparation tasks, specifying the actions that should be carried out which typically involve various ingredients and kitchen devices. In an IoT scenario, instructions in cooking recipes can form the basis for automatically controlling kitchen devices without any programming. However, as these instructions are written in natural language, they first need to be transformed or parsed into machine-interpretable commands. As a step towards this, we investigate methods for identifying the various types of actions (events) that kitchen devices are involved in. We cast this problem as a clustering task, whereby recipe instructions involving a given device of interest, are automatically grouped according to the type of event described. Each sentence in every instruction is represented by its embedding vector which is computed using a BERT-based model, specifically one pre-trained using a Roberta architecture. We cluster these sentence embeddings using our newly proposed interactive machine learning (IML)-based framework underpinned by the HDBScan clustering technique. We demonstrate that our IML framework can detect events in sentences with satisfactory accuracy, reaching almost the same level as human performance. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact This dataset was created by a couple of annotators and was used to evaluate an algorithm for clustering cooking oven and fridge commands. This work adds to the greater vision of having self programmed kitchen appliances and sensors to aid users in the kitchen. The users could benefit from being taught on how to cook better (i.e. nutrition, precision cooking) and reminding them of when their involvement is needed, particularly useful for people with dementia. 
URL https://figshare.com/articles/dataset/Interactive_Clustering_of_Cooking_RecipeInstructions_Towards_t...