The Deterrence of Deception in Socio-Technical Systems

We have assembled a team of computer scientists and psychologists to do breakthrough research on deception. This is not just the basic problem at the heart of cyber-crime, but is central to human behaviour. Deception is the flip side of cooperation; as our ancestors evolved to cooperate in larger groups, so also we evolved the ability to deceive - and to detect deception in others. This includes the ability to deceive ourselves, which in turn helps us deceive others.

The move of business and social life online is changing deception in many ways. Some of these are essentially mechanical: on the one hand it's easy for crooks to create good copies of bank websites, while on the other hand companies can collect and analyse ever more data to detect fraud. Other changes affect our behaviour; for example, as transactions become more impersonal, the inhibitions against cheating become less. It feels less wrong to defraud a website than to defraud a person, and as more commerce goes online, fraud is rising. There is a strong practical reason for finding ways of making transactions feel more personal again, and this is one aspect of our investigation.

But there are much bigger and deeper issues. Existing deception research has almost all dealt with the static case of whether the experimental subject could tell whether another person was lying. The answer is "usually not"; we're not good at detecting people telling lies in the laboratory where the stakes are low and there is no interaction. We will move this to a new level by studying how we can deter deception in interactive contexts. We have a long list of ideas we want to test, by building a framework in which people play games in conditions with players who are mechanical, anonymous, partly identifiable or socially connected, and where players can cheat but not punish, punish but not cheat, or both. We will explore conditions where "cheating" subjects believe they are breaking social norms against conditions where deception is socially acceptable, such as bluffing in online poker. We will do experiments in online interviewing to explore the circumstances in which subjects can detect deception interactively. Another thread will be exploring the extent to which surveillance, by humans, by software or both, can act as a deterrent to cheating. Finally we will investigate how all this relates to the perception of privacy, so we can better understand what forms of fraud surveillance are acceptable as well as effective.

The aim of our research is not merely to come up with better mechanisms and design principles for deterring deception at e-commerce websites, and indeed in social networks. It is to deepen our fundamental understanding of how deception works by exploring the different perspectives that online interaction creates, and allows us to manipulate. This will, at the level of basic science, enable us to understand better what it means to be human, and the potential for our humanity to be expressed, realised and developed in the complex socio-technical systems on which we are all coming to depend.

In addition to the researchers' immediate professional circle of researchers into security and deception, the beneficiaries of this project will include the following wider groups.

First, academics studying psychology and behavioural economics are keenly interested in the cognitive basis of decision-making in the presence of risk and in the context of adversarial behaviour. We indeed hope a systematic study of deception deterrence online may contribute important insights to our deep understanding of what makes us human, given the centrality of deception in much human behaviour and the fact that deception is changing as things move online. If we do win such insights, the benefits will spread beyond academia to the popular understanding of psychology, of behaviour and indeed of ourselves.

Second, firms conducting retail business online suffer fraud losses of typically 0.3% of sales and refuse a further 4% of offered sales because of alarms from their fraud-detection systems. We believe that better design for deception deterrence can reduce 'friendly fraud', that is, fraud conducted by relatives, work colleagues etc, by perhaps a third, and it's thought that friendly fraud might be a third of the total. This holds out the prospect of reducing fraud by a tenth, or 0.03% of online turnover. This will amount to several tens of millions for the UK and in the high hundreds of millions worldwide. Further gains may be in prospect if fraud detection systems can be tunes to have a better ROC as the loss due to sales foregone might be reduced. Yet further fraud savings may be possible in electronic banking systems.

Third, the insurance industry has a keen interest in reducing fraudulent claims, which our project will also tackle directly. The figures are not as precisely known in this sector as the ground truth is harder to measure; however we believe that insurance fraud might be reduced by broadly the same order of magnitude - namely tens of millions a year for the UK.

Fourth, the public sector should benefit from having tested methods available for using behavioural techniques to reduce tax and welfare fraud. In discussions with DWP it has become clear that they do not have the IT resource to conduct their own experiments but are keen to adopt techniques developed elsewhere.

Finally, policymakers will benefit from having a clearer understanding of the costs of cybercrime and the behavioural pathways available to mitigate it. It is a policy goal that the UK become the best place in Europe for e-business, yet we are some ways behind; the most recent Eurostat survey put us second last behind Latvia. If we can acquire world-leading expertise at deterring deception, this may go a significant way towards catching up.