Hetero-multi-modal stroke diagnosis and prognosis deep neural network model

In this piece of work, we are tackling the creating of a diagnosis and prognosis model specific for stroke that is multi-modal and hetero-modal with the core of the model is a generative one that has a highly morphologically descriptive latent representation. The latent representation will be used as a compressed representation of the imaging modalities and be combined with latent features extracted from embedding the neurological reports with the aim of diagnosis and prognosis for delirium. Lastly, we added some form of introspection to highlight what parts of each modality are the most important for the task at hand.