Novel Anomaly Detection Methods for Telecommunication Data Streams.

Lead Research Organisation: Lancaster University
Department Name: Mathematics and Statistics

Abstract

Anomaly detection is used in many places, and almost everywhere lots of data are processed, to answer questions like "is this transaction fraudulent?" or "do we need to switch off this expensive piece of machinery and do a maintenance check?" or "is there a planet orbiting this star?". The methods used to do this are complex and varied, and not all of them are fast enough to function well on streams of data that arrive in real time.

This PhD approaches the anomaly detection problem from a statistical standpoint. Instead of heavy models that require lots of training data and computational power, it looks at lighter-touch algorithms that can work well in applications where efficiency is important and detect anomalies in real time as soon as they develop. This involves testing, evaluating, and developing methods using both real and simulated datasets.

The project is sponsored by BT, and some of the problems they tackle are about flagging up issues in the telecommunications network that engineers need to go out and fix. These show up as strange blips in overall network usage over time against a backdrop of normal human behaviour (which can itself be very strange). Dealing with ways to distinguish anomalies from the varying structure in the data signal is one of the project's focus areas.

In partnership with BT.

Planned Impact

This proposal will benefit (i) the UK economy and society, (ii) our industrial partners, (iii) the wider community of non-academic employers of doctoral graduates in STOR, (iv) the scientific disciplines of statistics and operational research and associated academic communities, (v) UK doctoral students in STOR, and (vi) the CDT students themselves.

Below we outline how each of these communities will realise these benefits:

(i) The UK economy will gain a competitive edge through a significant increase in the supply and diversity of doctoral STOR professionals with the skills required to undertake influential, responsible and impactful research, and who have been trained to become future leaders. Our goal is that our future alumni who enter industry assume leading roles in realising the major impact that STOR can make in achieving effective data driven decision-making. Our existing alumni are already starting to achieve this. A wider societal benefit will accrue from research contributions to EPSRC Prosperity Outcomes, e.g. to the UK being a Productive and Resilient Nation.

(ii) Our industrial partners will particularly benefit from the skills supply identified in item (i), as likely employers of STOR-i graduates. They will further benefit from teaming with a community of leading edge STOR researchers in the solution of substantive industrial challenges. Mechanisms for the latter include doctoral projects co-supervised with industry, industrial internships, engagement in research clusters and industrial problem-solving days. Our training programme will give students the skills they need to ensure that research is conducted responsibly and that outcomes are successfully communicated to beneficiaries. The value that our industrial partners place on working with STOR-i can be seen through the pledged cash support of £1.7M.

(iii) A wider benefit will accrue from the employment of STOR-i graduates, equipped as described in items (i) and (ii), across non-partner public and private sector organisations. The breadth and depth of training provided by the CDT will enable students to quickly make a difference in these organisations, using their research skills to affect significant change.

(iv) The STOR academic community will benefit from methodological advances and from the increase and diversity in the supply of STOR researchers who value, and have experience of, collaborative research. Our alumni will be leaders in 21st Century Statistics with a strong culture of, and training in, reproducible research and a focus on achieving impact with excellence. Our recruitment strategy will further benefit this community in achieving a healthier supply of high-quality doctoral candidates from diverse backgrounds. Our research internship programme gives top mathematically able individuals from across the UK an experience of STOR research and has been shown to increase applications for STOR PhD programmes across the UK.

(v) Elements of the STOR-i programme will benefit the wider community of UK doctoral students in STOR. Using financial support from our industrial partners, we will continue our National Associate Scheme. This will provide up to 50 UK STOR doctoral students with funding and access to elements of STOR-i's training programme. An annual conference will provide opportunities for learning, networking and sharing research progress to members of the scheme.

(vi) STOR-i students will benefit from a personalised programme that will support each individual in fully achieving their research leadership potential, whether in academia or industry. Students will be given the tools and opportunities to develop research and broader skills that will enable them to achieve maximum scientific impact for their work. Our current alumni provide strong evidence that these future graduates will be extremely employable.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/S022252/1 01/10/2019 31/03/2028
2284300 Studentship EP/S022252/1 01/10/2019 30/09/2024 Kes Ward