The Effects of Information Dynamics on Trust and Reputation

Context

Modern systems are increasingly complex, feature multiple independent, decision-making agents. These agents including IoT, autonomous vehicles and human beings. A single system can include multiple types of agent. Agents aim to act within their best interests, requiring some mechanism for selecting who to interact with under uncertainty. Similar organic systems provide this in the form of trust. Trust is not novel to computer science with cryptography being the paradigm. However, conventional cryptographic methods are often infeasible or unsuitable for complex societies. As such, the modern conception of computational trust aims to capture the generalized phenomenon to describe existing systems and construct improved ones.

In highly populated environments, agents needs some way of establishing trust "off the ground". It is infeasible for an agent to safely interact with and judge every potential partner. Reputation is used to overcome this. Agents will share information about others throughout the network to build a richer idea of the community than they could alone. As such, the basis for trust on all such systems is message passing. It is easy to focus on the content of a message when trying to understand its impact. However, knowing when, where and how a message spreads is crucial to understanding its influence. There are many unexplored ways in which a malicious entity can exploit information flow in a network. Whether exploiting slow, faulty information propagation to "hide" from the consequences of their actions or exploiting fast but unverified information propagation to spread a message, information dynamics are a vulnerability of any reputation system. This project aims to look at some fundamental tenets of the study of information dynamics and its influence on multi-agent decision-making, with a focus on the exploitation and protection of individual agents and the community as a whole.

Research Methodology

The study of general computational trust is itself quite novel. Formal attempts at capturing this phenomenon are even newer. As such, to study a novel aspect of trust such as information dynamics would require at least the novel application of existing methods. Formal tools for analysing networks (network theory) and interacting, strategic agents (game theory) exist but their application to the modern conception of computational trust is sparse if at all present. As such, this project endeavours to draw from established formal methods from a variety of fields to tackle a currently unresolved but fundamental issue in our understanding of trust and social decision-making in complex systems. More precisely, the project aims for the following:

- Understanding how the structure of a network affects the movement of information on it and the impact of this on the members of the network.
+ This can be used to then construct networks which are less vulnerable to such issues.
- Analysing how malicious entities may exploit these dynamics (e.g. exploiting delays in information propagation to "hide" from consequences).
+ This can be used to inform the development of mechanisms to mitigate malicious behaviour on such networks.
- Analysing strategies of innocent parties that are more or less likely to make them vulnerable to the endeavours of a malicious entity.
+ In the case of engineered systems, this can be used to construct more robust agents.
+ In the case of social networks, this could inform education and policy decisions around human actors (e.g. when trusting other individuals, websites or news sources online).

EPSRC Strategies
Mathematical Sciences; Global Uncertainties; Digital Economy

Collaborators
While there are no formal collaborators as of yet, some companies have expressed their interest in the ramifications and outcomes of this work. Talks are ongoing.

It is part of the nature of Cyber Security - and a key reason for the urgency in developing new research approaches - that it now is a concern of every section of society, and so the successful CDT will have a very broad impact indeed. We will ensure impact for:

* The IT industry; vendors of hardware and software, and within this the IT Security industry;

* High value/high assurance sectors such as banking, bio-medical domains, and critical infrastructure, and more generally the CISO community across many industries;

* The mobile systems community, mobile service providers, handset and platform manufacturers, those developing the technologies of the internet of things, and smart cities;

* Defence sector, MoD/DSTL in particular, defence contractors, and the intelligence community;

* The public sector more generally, in its own activities and in increasingly important electronic engagement with the citizen;

* The not-for-profit sector, education, charities, and NGOs - many of whom work in highly contended contexts, but do not always have access to high-grade cyber defensive skills.

Impact in each of these will be achieved in fresh elaborations of threat and risk models; by developing new fundamental design approaches; through new methods of evaluation, incorporating usability criteria, privacy, and other societal concerns; and by developing prototype and proof-of-concept solutions exhibiting these characteristics. These impacts will retain focus through the way that the educational and research programme is structured - so that the academic and theoretical components are directed towards practical and anticipated problems motivated by the sectors listed here.