Automation and Contemplation for Model Adaptation in Multiagent Interactions

An agent is a computer system that acts intelligently given its sensory input from the environment. Agent technologies have proved to be effective and reliable solutions in many practical applications and will continue to play a major role in modern society. For example, the eBay buyer agent recommends good deals for people in an e-market. The Google self-driving car operated by an autonomous agent has successfully navigated thousands of miles on the road. A smart meter controlled by an intelligent software agent helps optimize energy consumption for a household.

In many such applications, an autonomous agent (namely a subject agent) is expected to make a rational decision by predicting behaviors of other agents in a common environment. The decision quality relies on building decision models of the other agents and then solving the models to understand how the other agents will behave in the environment. When the subject agent's model is deployed in a real-world application, it may fail since the subject agent may receive unexpected observations incurred by other agents. Hence the challenge is about the prediction of other agents' behavior and the interpretation of model failure so as to adapt the subject agent's model for successful interactions.

The goal of this project is to improve the subject agent's adaptation by automating the model construction of other agents and revising its own decision model when the model fails in the execution. This project will propose scalable learning algorithms to build decision models of other agents upon historical data of agents' interactions. The algorithms will also facilitate the model construction in a new problem domain that will be likely larger and more uncertain in practice. To interpret failures of the subject agent's decision model, this project will search for a novel reasoning technique to identify the most probable reasons behind the failures, and accordingly revise the model so that the subject agent's decisions can be adapted to the other agents' behaviors in their real-time interactions. This project will implement all the proposed techniques in a toolkit and conduct comprehensive tests to evaluate practical utilities of the toolkit. Real-world applications on personalized learning and intelligent computer game AI engine development will be extended through our industrial collaborators.

The broader impact of this research will be to enable individual agents to act rationally in complex multiagent environments. This is a crucial step toward the integration of autonomous agent technology within society that will support humans in tasks such as disaster response, energy distribution and security operation.

The proposed research will benefit a wide range of industrial stakeholders that either develop their core technologies on AI or improve their services through AI related products. It would particularly facilitate the development of intelligent agent technologies in a real-world application where the agent needs to optimize its decisions when interacting with other agents. The agent could be either a piece of software, a computer system, or even a human being.

This project will build a data-driven multiagent decision making toolkit and test the practical utilities of the toolkit in the Starcraft gaming systems using real-word game replay data. The integration of the toolkit into the gaming systems will provide insights to computer game sectors on using data-driven approaches in a game AI engine, which has not been much explored in the game industries.
As a data-driven approach does not demand much effort from problem domain experts on the modeling task, it can be easily adopted by many industrial sectors that do not have a good background in intelligent technologies.

This project will focus on two particular applications through our existing collaboration with external industrial partners. The first application is to facilitate the development of a personalized learning platform in the TWI training centre where the learning data will be used to construct a student model so as to configure learning pathways for individual students. This application will help students to improve their learning outcomes while reducing the workload of trainers on preparing a diverse set of training materials.

The second one is to develop intelligent non-player characters (NPCs) in a game AI engine in the mobile game company - SinceMe. The proposed research will exploit existing game data to learn typical behaviors of human players. The learned behaviors will inform a better design of NPCs so that the NPCs can adapt their strategies to how human players act in the changing game environment. The application will not only enhance players' experience in the gameplay but also improve the game development efficiency in SinceMe.

We are also seeking for more industrial collaboration in order to extend the impact of our research outcomes in this project. For example, in a healthcare application, the proposed agent technologies could be used to build a human-like assistant robot that provides daily services for the elderly. The PI of this project, as a co-lead of the University Grand Challenge in Health and Wellbeing, will drive the development of such an application through the Tees Healthcare Innovation Partnership between Teesside University and South Tees Hospitals NHS Foundation Trust.

The toolkit will be available in a project web site and its utility will be demonstrated at conferences, e.g. the prestigious IJCAI and AAAI conferences, where potential industrial partners are present. We will communicate our research findings with relevant stakeholders through traditional mass media, like newspapers, television, radio and popular magazines. This will be organized through our partnership with Native Consultancy, which supports the targeted dissemination of the University's research, and has recently worked with the PI. Social media platforms like Twitter, YouTube and LinkedIn, will be also used to further enhance the outreach of the project and specifically target and connect end-user audiences. While giving regular lectures in University open days and delivering public talks organized by Digital Catapult Centre North East & Tees Valley or others, we expect that the successful applications of our research outcomes will also raise public awareness of AI technologies and increase their acceptability in a modern society.