Sequential Decision Making in Real-time Digital Advertising

Digital marketing has been transformed by the development of real-time bidding platforms. These platforms allow advertisers to decide sequentially whether to place a bid on a particular advertising location (e.g., a website), how much to bid on and who to bid on. The speed of the markets means that these decisions need to be taken almost instantaneously, which has led to many algorithmic approaches being used. Another key aspect of decision-making in the digital advertising world involves the balance between privacy and utility. The ideal situation for the marketer is to have access to the user's entire online browsing history whereas the ideal set up for an internet user is for the marketers to have minimal access to their information. This leads to a trade-off between what is the most affordable and possibly relevant browsing experience a user can have with the information they are willing to share with the marketers. The nuances of the real-time bidding problem mean that directly applying existing algorithms for sequential decision making may lead to sub-optimal decisions being taken. Therefore, the project will involve designing and analysing new algorithms tailored to specific settings in digital advertising, including privacy preserving learning. This has become extremely important and almost an existential threat for many advertising technology companies in recent time due to the deprecation of "cookies" (most popularly used digital identifier used to track user's online activity) by Google Inc. Without access to an identifier, how does one deliver a personalized privacy preserving browsing experience for an internet user? In this project, we aim to design new algorithms that draw ideas from reinforcement learning algorithms that deal with sequential decision-making with uncertainty, semi supervised learning algorithms that use aggregate information or partially labelled data to learn and optimize policies and active learning methodologies that focus on how to learn efficiently with limited samples, particularly when labelling data is expensive.
This project falls within the EPSRC research themes of "Operational Research", "Statistics and Applied Probability" and "Digital economy" and aligns with EPSRC's strategic priority related to "artificial intelligence, digitisation and data: driving value and security". This research is being done in collaboration with "Imperial College, London" and "The Trade Desk", which is a multinational company that specializes in real-time programmatic marketing automation technologies.
Towards this, our goals are two-fold: a) Design rigorous algorithms with provable guarantees b) Demonstrate the efficacy of our algorithms on real data and drive towards real world impact.

The primary CDT impact will be training 75 PhD graduates as the next generation of leaders in statistics and statistical machine learning. These graduates will lead in industry, government, health care, and academic research. They will bridge the gap between academia and industry, resulting in significant knowledge transfer to both established and start-up companies. Because this cohort will also learn to mentor other researchers, the CDT will ultimately address a UK-wide skills gap. The students will also be crucial in keeping the UK at the forefront of methodological research in statistics and machine learning.
After graduating, students will act as multipliers, educating others in advanced methodology throughout their career. There are a range of further impacts:
- The CDT has a large number of high calibre external partners in government, health care, industry and science. These partnerships will catalyse immediate knowledge transfer, bringing cutting edge methodology to a large number of areas. Knowledge transfer will also be achieved through internships/placements of our students with users of statistics and machine learning.
- Our Women in Mathematics and Statistics summer programme is aimed at students who could go on to apply for a PhD. This programme will inspire the next generation of statisticians and also provide excellent leadership training for the CDT students.
- The students will develop new methodology and theory in the domains of statistics and statistical machine learning. It will be relevant research, addressing the key questions behind real world problems. The research will be published in the best possible statistics journals and machine learning conferences and will be made available online. To maximize reproducibility and replicability, source code and replication files will be made available as open source software or, when relevant to an industrial collaboration, held as a patent or software copyright.