Deep Generative Models for Multimodal Data with Applications in Medicine

Deep generative models (DGMs) have demonstrated significant potential in tackling the complexities of modern datasets, yet effectively integrating multiple data modalities remains a substantial challenge. Currently, there is no standard approach for managing the diverse structures, scales, and interactions inherent in multimodal data, making it an active and evolving area of research. In the medical field, patient data is often collected from a range of sources, including electronic health records (EHRs), medical imaging, physiological measurements, and genetic information. While each modality provides valuable insights, their varying formats and structures complicate efforts to analyze them in a cohesive, holistic manner. The primary goal of this DPhil project is to develop deep generative models capable of learning from and integrating these multimodal medical datasets, with the aim of advancing healthcare diagnostics, prognosis, and personalized treatment recommendations.
Multimodal medical data presents a unique challenge due to its high dimensionality and heterogeneity. For instance, EHRs consist of structured categorical data such as diagnoses, prescriptions, and lab results, while imaging data, such as MRIs or CT scans, is high-dimensional and unstructured. Genomic data adds yet another layer of complexity, often represented as sequences or large-scale expression profiles. The primary objective of this project is to develop and extend deep generative models to capture the shared and modality-specific information, allowing for a more comprehensive understanding of patient health and disease progression.
One of the core challenges this project will address is handling missing or incomplete data, a common issue in healthcare datasets. Not all patients have data available across all modalities, which makes effective model learning difficult. A key focus of this research will be the design of methods that can account for missing modalities, either by imputing missing data or by developing models that remain robust in the face of incomplete information.
Another significant aim of the project is to enable realistic data synthesis. DGMs can be used to generate high-quality synthetic medical data, which is particularly valuable in scenarios where privacy concerns or limited sample sizes make it challenging to work with real patient data. By generating realistic multimodal patient profiles-complete with imaging data, clinical records, and genetic information-these models can be used to simulate different disease trajectories or outcomes. This capability will be instrumental in augmenting training datasets for machine learning models or supporting diagnostic tools, allowing for better model generalization and robustness.
The models developed in this project will be tested on a variety of publicly available medical datasets, such as the UK Biobank, which contain a wealth of multimodal data, including EHRs, imaging, and genetic profiles. By addressing the challenges of multimodal integration and incomplete data, this project aims to push the boundaries of what is possible with AI in healthcare, contributing to the development of more intelligent and robust clinical decision-making tools.