Differential privacy: real-world implementations

Lead Research Organisation: Imperial College London
Department Name: Dept of Computing

Abstract

My project aims at designing Privacy-Enhancing Technologies that allow data analysts to draw important insights from large behavioural datasets without disclosing sensitive information. Part of the project includes finding vulnerabilities in privacy guarantees enforced by existing technologies.

Research area: Privacy and security

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/N509486/1 01/10/2016 31/03/2022
1958603 Studentship EP/N509486/1 01/10/2017 31/03/2021 Andrea Gadotti
 
Description Data-driven decision-making is useful for business, researchers and policy-makers. However, in order to be analysed, the data has to be collected and shared with those who want to use it. This raises legitimate privacy concerns.
Traditionally, the standard tool to share data while protecting privacy has been anonymization. However, research shows that anonymization is not enough to protect modern datasets. To address these limitations, researchers and companies are proposing new privacy-enhancing technologies.
Diffix is a commercial tool for privacy-preserving data analysis developed by the company Aircloak. It has been proposed as a novel query-based mechanism satisfying alone the EU Article 29 Working Party's definition of anonymization.
In a paper accepted into USENIX Security '19, I and my coauthors present a new class of noise-exploitation attacks, exploiting the noise added by the system to infer private information about individuals in the dataset. Our attacks demonstrate that adding data-dependent noise, as done by Diffix, is not sufficient to prevent inference of private attributes.
Exploitation Route The attacks I and my coauthors published against Diffix provide important insights for the development of robust privacy-preserving mechanism for the safe use of data, such as those relying on differential privacy. Specifically, these attacks are the first to prove on a deployed system that data-dependent noise addition - while appealing from a utility standpoint - is risky for privacy. Future research can start from these attacks to move into at least two directions. First, the attacks can be refined, extended and generalized to other systems. Second, the attacks offer a clear indications of vulnerabilities to avoid in the development of new technologies for privacy-preserving data analysis.
Sectors Digital/Communication/Information Technologies (including Software)

 
Description In a paper published in USENIX Security '19, I and my coauthors propose a new class of noise-exploitation attacks against Diffix, a commercial tool for privacy-preserving data analysis. Aircloak, the company which develops and commercializes Diffix, assigned severity "Very High" (highest to date) to the vulnerability we found and released a patch for the system to mitigate the risk that it can be exploited.
First Year Of Impact 2019
Sector Digital/Communication/Information Technologies (including Software)