Machine Learning for Game World Generation using Intrinsic Motivation as an Objective Function.

New forms of machine learning are increasingly important in games research, and generative models demonstrate great potential for content creation. Gatys et al. [1] demonstrate a method for texture synthesis using convolutional neural networks that exceeded the state of the art in the fidelity of of textures generated. Chen et al. [2] develop a method for high definition photographic image rendering from semantic pixel labels using reinforcement learning. Thompson et al. [3] train a convolutional neural network to model complex fluid simulations, allowing them to approximate fluid simulations more efficiently. Bansal et al. [4] train reinforcement agents in competitive self-playing environments resulting in the development of complex emergent behaviours in otherwise simple environments. When such systems are computationally efficient enough to render in real time, they will transform the way video games are both produced and played.
Variational, adversarial and autoregressive systems can be very successful at performing inference on the statistical distribution of high dimensional data. This allows models to generate content with sophisticated representations. However, they are not very good at novel content generation.
Schmidhuber et al. [5] propose a Reinforcement Learning agent, motivated not by external reward, but by intrinsic motivation. The agent is driven by Compression Progress. It constantly samples the world, trying to compress it, inferring patterns and regularities in the data, but motivated to seek out data that is novel "as long as the algorithmic regularity that makes it simple has not yet been fully assimilated by the adaptive observer who is still learning to compress the data better", which they define as 'the time-dependent subjective Interestingness'.
I will develop new kinds of machine learning systems driven by intrinsic motivation (such as time-dependent subjective interestingness [5]), optimised towards creating systems that display complex emergent behaviour. Defining the structure and complexity of emergent behaviour is inherently subjective [6]. An agent that has this subject measure of novelty - with respect to other structures in an underlying system - and intrinsic motivation to seek it out (in the form of an objective function), should be able to detect or encourage the development of complex systems that produce more novel outcomes.
Research Questions
- Can Intrinsic Motivation be used to develop efficient procedural content generation systems that can be deployed in live game environments for texture, model, and world generation?
- Can Intrinsic Motivation be used to develop agent behaviour policies or physical simulations that behave in complex, unpredictable ways?
- What kinds of machine learning techniques are best suited to implementing this approach? (Reinforcement learning, variational inference, differentiable neural computers)
- Can this technique be augmented with existing generative systems to encourage sampling more interesting content?
Research Plan I will begin by designing systems that use objective functions to create agents driven by compression progress. These will then be applied to problems of procedural content generation and complex agent behaviour. At the end of this process I will create a game that implements and demonstrates these methods for procedural content generation, driven by machine learning.
[1] Gatys, Leon, Alexander S. Ecker, and Matthias Bethge. "Texture synthesis using convolutional neural networks." In Advances in Neural Information Processing Systems, pp. 262-270. 2015.javascript:WebForm_DoPostBackWithOptions(new%20WebForm_PostBackOptions("ctl00$oSaveBar$btnSave",%20"",%20true,%20"",%20"",%20false,%20true))
[2] Chen, Qifeng, and Vladlen Koltun. "Photographic image synthesis with cascaded refinement networks." In The IEEE International Conference on Computer Vision (ICCV), vol. 1. 2017.
[3] Tompson, Jonathan, Kristofer Schlachter, Pablo Sprech

The IGGI Centre for Doctoral Training will impact upon:

The Digital Games Industry: IGGI will inject a substantial cohort of 55+ PhD graduates and a wide range of academic research leaders with direct experience of research collaboration with the UK digital games industry. Although large, the UK games industry is fragmented and geographically dispersed, consisting primarily of SMEs. Increasing skill levels and injecting research advances in such a community is best achieved through employment of and engagement with creative and entrepreneurial PhD graduates with good communication skills, and through stable long-term government-funded collaborative projects which offer the opportunity for research engagement at a time to suit the business cycles of games industry partners. IGGI offers the opportunity for a step change, yielding increased profits through an internationally distinctive UK games industry which is technologically advanced and research-aware. The financial barriers to starting a company in this area are low and many IGGI graduates will start their own games businesses, mentored by experienced investors and entrepreneurs, significantly increasing their chances of creating a successful games enterprise. Data mining tools developed during IGGI will allow increased understanding of game players, which can increase profitability of mainstream games.

Parents, Game Players and Wider Society: Large and growing numbers of people are playing digital games with unprecedented enthusiasm. In a recent Forbes magazine article it was suggested that, by the age of 21, the typical child has played an average of 10,000 hours of digital games. Creating games which engage a wider range of players and which increase the social and scientific value obtained through playing games can have massive benefits: both economic ones and ones which harness the massive "cognitive surplus" implied by game players who are clocking up thousands of game hours. The potential benefits here are cultural (e.g. to raise awareness in important areas such as environmental change), scientific (e.g. to conduct experiments which use artificial economies to test economic theory), social (e.g. to educate children about science) and therapeutic (e.g. to use games to increase mobility in the elderly).

Scientists: Gameplay data can provide information about human behaviour and preference on a massive scale - this provides a major new experimental tool for researchers in Economics, Ecology/Biology, Computer Science, Psychology, Mathematics, Media and others. The very recent announcement (20th June) of a proposed call in the EU Horizon 2020 research funding programme on "Advanced digital gaming/gamification technologies" underlines how much the EU values this area and the opportunities for pan-European research in games and sustainability for IGGI.

IGGI Graduates and Supervisors: Digital games are already a major attractor to computer science and digital media courses. IGGI will provide a beacon for innovation in digital games, with heavy competition for PhD places allowing recruitment of top students. For each IGGI graduate, learning and conducting research alongside a strong cohort of students with related but different interests and expertise, with extensive interaction with industry, will give rise to a highly rounded and employable PhD graduate, who will be highly sought by both UK games industry and the growing games research community. Supervisors will gain knowledge at the cutting edge of games and gamification research.

Through the CDT, IGGI investigators, supervisors and students will become well versed in the issues and techniques of the digital games industry, developing a long-term understanding which will, we believe, result in a stronger digital games industry, a wealth of fascinating new research questions, and real benefits for wider society through the now-ubiquitous medium of digital games.