Mining the Network Behaviour of Bots

The botnets phenomenon has quickly become a major security concern for all the
Internet users. In fact, not only has it rapidly gained popularity among the
mass media, but it has also received the attention of the research community
interested in understanding, analyzing, and detecting bot-infected machines.
Once infected with a bot, the victim host joins a botnet, a network of
compromised machines that are under the control of a malicious entity. Botnets
are the primary means for cyber-criminals to carry out criminal tasks,
such as sending spam mail, launching denial-of-service attacks, or stealing
personal data such as mail accounts or bank credentials.

Clustering and correlating network events represent the state-of-the-art when
it comes to detecting and understanding the botnets phenomenon from a network
perspective. While effective, such approaches rest on weak foundations being
vulnerable to easy-to-perform (time and network) obfuscation attacks.

The goal of this project is to build on the promising results of our previous
work to explore novel machine-learning techniques to make the state-of-the-art
more accurate and robust against evasions and advanced malware. Exploring the
possibilities of advanced malware (and thus bots) to enable the development of
novel mathematical techniques to address such threats is not a mere academic
exercise. On the contrary, it is of paramount importance to build robust and
hard-to-elude mitigation approaches; something we currently lack, as
acknowledged by the research community at large.

On the cyber security side, we will develop techniques to analyze the network
traffic generated by a bot sample. Our analysis will focus on inferring the
interesting part of a bot's network behaviour to automatically generate models
that faithfully describe it. Our analysis aims at being independent from the
underlying botnet infrastructure, payload-agnostic, and able to pinpoint
legitimate-resembling malicious activities. The network flows of a monitored
bot will be initially filtered to remove well-defined attack patterns. The
remaining flows will be clustered using a number of network features and
suitable similarity functions. Clusters whose size exceeds a given threshold
will then be analyzed for periodicity: bots tend to engage in similar network
activities that have interflow intervals that either are sampled independently
from a potentially unknown probability distribution, or belong to a small
number of well-defined clusters. Once clusters exhibiting interesting
periodicity patterns are identified, they can be used, along with their network
features, for detecting (or understanding the behaviour of) bots in a mixed
population containing both compromised and clean hosts.

On the machine learning side, we propose to explore the use of conformal
prediction developed by our team to make such cluster-based analysis more
accurate and robust against arbitrary obfuscation-based evasion attacks. A
powerful clustering method is based on nonparametric probability density
estimation. A recent work proposes a computationally efficient method of
nonparametric density estimation based on conformal prediction and inherits its
properties of validity. We plan to explore the use of this method for the
purpose of robust clustering. A theoretical challenge is to spell out and study
the properties of robustness for this clustering method that are inherited from
the validity of the study mentioned above. In addition, the property of
validity of conformal predictors is usually established under the randomness
assumption; we will explore how this assumption can be relaxed. In addition,
the property of validity can be used to control the number of "alarms"
(predicting that a host is compromised) raised by a bot detection algorithm.
This is valuable in situations where alarms have to be investigated by human
experts but the available manpower is limited.

This project is concerned with the development of novel
obfuscation-resistant techniques to analyze and detect core network
behavioral traits of bot-infected computing devices.

We will organize a two day workshop on the subject of machine learning
and malware analysis. The workshop will be aimed at bringing together
all the project collaborators, academic researchers and industry
practitioners with interest in the project's topics. The goal of the
workshop is to narrow the gap that nowadays exists between security
research carried out in academia and industry to face common threats.

The workshop will be held at the end of the second year of the project
and members of the advisory board (see next) will be encouraged to
attend. The workshop will include a number of invited talks from
representative of research industry laboratories, including---but not
limited to---McAfee Labs (UK), Symantec Research Labs (Europe), VU
University Amsterdam (The Netherlands), and FORTH-ICS (Greece), for
which this project has already received strong letters of support.

We will also appoint a panel of experts---mostly from the
industry---who will provide advice on practical problems and needs,
and help to promulgate the results of our research.

The Information Security Group at Royal Holloway, University of London
has considerable experience in working with industry. Over the last
ten years it has convened expert panels on public key infrastructures
(the "PKI Club"), authentication and identity management (the "AIM
Club") and, most recently, on cyber security. These panels,
comprising experts from industry and government circles, meet
regularly to discuss the challenges facing industry and government,
focusing on a different aspect at each meeting. We anticipate that the
experts advisory board for this project would operate in a similar way.

The composition of the board will be determined in the first few
months of the project. At least one member of the advisory board will
be invited to give an industry-centered talk at the workshop. The
board will meet four times, once within six months of the start of the
project and at the end of each year of the project.

We are confident our project will produce key results to overcome
advanced malware, which are of paramount interest to academic and
industry security researchers. However, not only the developed machine
learning techniques will improve our understanding of bot-related
threats, but they will also likely be applicable to other important
areas, such as System Biology, where time-series data---and clustering
in general---has to be examined.