Abstract Forward Models for Modern Games

The games industry is one of the fastest-growing industries in the world, with yearly revenues expected to increase from US$ 138bn in 2018 to US$ 180bn in 2021. The UK games industry is a worldwide leader that contributes significantly to wealth creation and export, with a clear growing tendency: 62% of the 2261 companies in the UK were founded in the last 8 years. Employing 12,000 people with sales valued in £4.3bn for 2017, this industry is the second largest market in Europe and the fifth in the world.

Games have also been excellent benchmarks for the advancement of AI. One of the most clear and recent examples of this is the progress on search methods in the game of Go. Go is a thousand years old board game of simple rules but complex strategy, where humans had dominated computer AIs since the beginning of the field. Monte Carlo Tree Search (MCTS), an AI technique that explores the different branches of actions that both players can take, became in 2016 the standard algorithm for creating Go AI players, giving birth to substantial research on variations and applications of this algorithm. Since then, MCTS has been used in thousands of other works in and outside games. This progress reached another milestone when Google Deepmind's Alpha Go mastered this game with a combination of MCTS and Deep Learning (DL).

MCTS uses a forward model (FM), which is a representation of the game state that allows to roll the state forward after applying any action in the game. This "simulator" is also used by other Statistical Forward Planning (SFP) methods that are also showing similar promise to MCTS in some domains, such as Rolling Horizon Evolutionary Algorithms (RHEA). It is however striking that despite the popularity and progress on SFP methods, they have barely reached the games industry. The most known uses of MCTS for Opponent AI in the games industry are in the Total War series by Creative Assembly, AI Factory on card games and Lionhead's tactical planning for Fable Legends. Given that the games industry is one of the fastest growing industries in the world and UK one may wonder why one of the top algorithms on AI in Games barely reaches far less than 0.01% of this industry.

The aim of this project is to incorporate an FM library into a modern games engine in order to facilitate research on the use of SFP techniques in large, complex, video-games. On the one hand, the project will address the technical and design problems of integrating a customisable FM that determines which elements of the real game state form part of the FM and how abstractions can be made. On the other hand, the project will aim to understand how SFP methods perform under these conditions in complex and large commercial-like games, investigating how these can be improved. The resultant framework will allow to test these methods in a wide range of games, with a special emphasis on proposing a Game AI competition for industry and researchers. Dissemination of the project's research outcomes will be guaranteed via open source libraries, frameworks, documentation and scientific papers.

This project builds naturally on the PI's recent work on GVGAI (for which he is main developer, organiser and coordinator of the competition, tracks and team - www.gvgai.net), and it proposes a step change on General Game AI research and its relevance to the games industry, adapting it to modern games. This project addresses directly the applicability of well-established methods such as MCTS/RHEA to large and complicated games and also the industry needs for fast, reliable and state of the art AI techniques. Our strong group of game industry partners (Microsoft Research, AI Factory, Bossa Studios, Creative Assembly and Gwaredd Mountain) will help steer the project into the interests of the game research and industry communities. Applications beyond games will also be explored with the help of our non-game industry partner (the Defence Science and Technology Laboratory).

The SCIENTIFIC COMMUNITY will benefit from this research. The AI in games community will benefit from having the possibility of testing their methods in commercial-like games. Having forward models in a large-scale games engine allows for research, across multiple game genres, the application of Statistical Forward Planning methods in complex environments. Researchers will benefit from being able to easily incorporate off-the-shelf advanced AI methods for non-player characters in complex game and non-game environments.

A project page to disseminate the findings and download the software, documentation deliverables and papers produced will be set up. Data from game logs for authorial analysis and explainability will be provided to build a large-scale database of high-quality data for new behavioural insights. Papers will be submitted to high impact factor journals on games (IEEE Transactions on Games) and evolutionary computation (IEEE Transactions on Evolutionary Computation), and also presented at most relevant conferences such as IEEE Conference on Games (CoG) or the AAAI Artificial Intelligence for Interactive Digital Entertainment (AIIDE). Special sessions, workshops and tutorials will be proposed for these venues. The international competition proposed will provide a great exposure of the project and the research breakthroughs achieved.

The VIDEO-GAMES INDUSTRY is a primary beneficiary of this proposal. The work performed will enable game companies to embed the latest AI methods in complex games, producing more interesting, challenging and fun games. The availability of stronger and better AI will reduce development time and costs and ease the automation of game evaluation and testing. By bringing the latest research to the industry, this project means a step change in the ways AI is applied in games, taking UK to a leading position in this industry. The hosting institution, QMUL, is part of the EPSRC IGGI CDT (EP/L015846/1) and its recent renewal. The goals and strategy of both IGGI and the Game AI group are closely aligned with this project: bringing the latest research in games to the industry.

Impact will be achieved by developing and providing an open source software library for an industry-standard game engine. The AI algorithms developed during the project will be publicly available on the project website. Given the popularity of the games engine and programming language within the games developers community, the reach of the implemented software is large. A direct impact mechanism is the possibility of an internship of the PhD student in Creative Assembly, one of the project partners, where our research findings can be applied. We'll present and promote our work at conferences with strong industry presence, such as Develop, the London Game AI Meetup and CoG.

The EDUCATION PROGRAMMES, concretely Computer Games and Computer Science degrees with modules on games and AI, will benefit from this research. Final year and MSc projects, and PhD topics will be able to use our framework for further research on Game AI. The PI has experience in this model, as his previous benchmark (GVGAI) has been used for teaching and as a base for student projects. GVGAI also showed that the proposed competition is a great asset for education, with multiple students across the globe participating in the challenges as part of their modules or projects. The integration of the framework with a game engine will aid the creation of more interesting games at game jams.

The project will produce interest among the GENERAL PUBLIC and media. The project will be promoted in showcase events, general interest articles (i.e MCV/Develop/ The Conversation) and regular press releases. The work will be promoted in public lectures and university open days. The team will also actively participate in events or magazines dedicated to the dissemination of research to the general public, such as Pint of Science nights or the CS4Fun Magazine.