Scalable Bayesian Reinforcement Learning in the Games Industry

One of the main challenges in reinforcement learning is identifying good data sampling strategies that effectively balance between exploring the space of all possible policies, and exploiting the trajectories that have yielded better outcomes so far. In environments with complex state and action spaces, such as those in video-games, this challenge becomes more apparent with traditional reinforcement learning algorithms often suffering from sample inefficiency, model bias, and over-fitting. Through incorporating uncertainty estimation and prior knowledge into the learning process, Bayesian reinforcement learning naturally balances this exploration-exploitation trade-off, making it a natural candidate for application in these environments. However, Bayesian reinforcement learning algorithms are more computationally intensive, which has hindered their wide-spread adoption. The proposed study will investigate methods to scale Bayesian reinforcement learning to handle large-scale problems, while maintaining computational efficiency and accuracy. On successful completion, this study will result in more efficient and stable training of reinforcement learning agents in the games industry. Through this, reinforcement learning algorithms can be more easily integrated into the design pipelines, resulting in quicker and more stable development as well as enhanced user experience.