Designing Interaction Freedom via Active Inference

Problem:
Reliable design of interactive systems using advanced sensors and machine learning (ML) is an unsolved problem. New sensors could expand how we interact with computers, but are still hard to design for, without overly constraining user behaviour.

AI algorithms can reduce human workload, but can fail in complex contexts and can control, deskill and dis-empower people. We have no principled workflows for designing interaction to allow users to flexibly share autonomy with supporting AI.


Objectives:
Integrate Active Inference theory into the human-computer interaction loop, linking human behaviour via sensors and ML/inference embeddings with dynamic mediating mechanisms to create end-to-end mutually adaptive loops between humans and systems.

Develop novel interaction mechanisms for explicit and implicit control of AI autonomy levels to empower people via shared autonomy, while maintaining their agency.

Create systematic, composable software tools for computational interaction design which support prototyping and analysis of ML-infused sensors coupled with humans, which can integrate probabilistic causal models to solve inverse problems with advanced sensors, adapting to closed-loop data.

Impact:
Using ML to give users freedom to express themselves individually, we can be robust to user heterogeneity, ensure fairness for diverse users and enable creative uses of technologies. Our tools will form the foundation of future usable interfaces with novel sensors and rich data spaces, and advances can be shared and rapidly built on, transforming HCI research workflows.

Applications:
1. Whole hand touch interaction via soft, programmable electronic skin, for personaliseable interfaces with novel forms, and prosthetics.
2. Google's Soli radar for 3D gesture, pose and proxemic interaction,
3. Radio Frequency sensing of humans for context-aware interaction.