Differential privacy: real-world implementations
Lead Research Organisation:
Imperial College London
Department Name: Computing
Abstract
My project aims at designing Privacy-Enhancing Technologies that allow data analysts to draw important insights from large behavioural datasets without disclosing sensitive information. Part of the project includes finding vulnerabilities in privacy guarantees enforced by existing technologies.
Research area: Privacy and security
Research area: Privacy and security
Organisations
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509486/1 | 01/10/2016 | 31/03/2022 | |||
| 1958603 | Studentship | EP/N509486/1 | 01/10/2017 | 31/03/2021 | Andrea Gadotti |
| Description | Data-driven decision-making is useful for business, researchers and policy-makers. However, in order to be analysed, the data has to be collected and shared with those who want to use it. This raises legitimate privacy concerns. Traditionally, the standard tool to share data while protecting privacy has been anonymization. However, research shows that anonymization is not enough to protect modern datasets. To address these limitations, researchers and companies are proposing new privacy-enhancing technologies. Diffix is a commercial tool for privacy-preserving data analysis developed by the company Aircloak. It has been proposed as a novel query-based mechanism satisfying alone the EU Article 29 Working Party's definition of anonymization. In a paper accepted into USENIX Security '19, I and my coauthors present a new class of noise-exploitation attacks, exploiting the noise added by the system to infer private information about individuals in the dataset. Our attacks demonstrate that adding data-dependent noise, as done by Diffix, is not sufficient to prevent inference of private attributes. |
| Exploitation Route | The attacks I and my coauthors published against Diffix provide important insights for the development of robust privacy-preserving mechanism for the safe use of data, such as those relying on differential privacy. Specifically, these attacks are the first to prove on a deployed system that data-dependent noise addition - while appealing from a utility standpoint - is risky for privacy. Future research can start from these attacks to move into at least two directions. First, the attacks can be refined, extended and generalized to other systems. Second, the attacks offer a clear indications of vulnerabilities to avoid in the development of new technologies for privacy-preserving data analysis. |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Description | In a paper published in USENIX Security '19, I and my coauthors propose a new class of noise-exploitation attacks against Diffix, a commercial tool for privacy-preserving data analysis. Aircloak, the company which develops and commercializes Diffix, assigned severity "Very High" (highest to date) to the vulnerability we found and released a patch for the system to mitigate the risk that it can be exploited. In a paper published in the 2019 IEEE International Conference on Big Data, I and my coauthors propose OPAL (for OPen ALgorithms), an open-source, scalable, and privacy-preserving platform for location data. OPAL is currently being piloted in Senegal and Colombia. |
| Sector | Digital/Communication/Information Technologies (including Software) |