Hierarchical Deep Representations of Anatomy (HiDRA)

Understanding the functions of genes in animal models such as mice allows researchers to learn about their roles in human disease. High-throughput phenotyping is used to conduct broad assessments of gene function through a combination of qualitative and quantitative assays, which seek to measure or visualise specific anatomical structures or organ systems. It is essential to understanding genotype-phenotype relationships and has guided the development of therapeutic targets for developmental, cardiovascular, neurodegenerative, and sensory disorders. Skeletal phenotyping is particularly crucial from a public health standpoint, with musculoskeletal disorders being responsible for 12% of all general practitioner visits in the UK at a cost of 10.8 million working days and some £4.7 billion to the NHS per year.

Broad assessments of the skeleton are particularly laborious and subjective due to its anatomical complexity and the range of potential anomalies one might observe. In mouse phenotyping, plain x-ray images are routinely acquired from multiple viewpoints, orientations and scales to ensure complete coverage of the whole animal. Phenotypes are identified through manual inspection by domain experts, which is prohibitively time-consuming to perform at scale. The International Phenotyping Consortium (IMPC) comprises eight institutions that collect x-ray images of mice and annotate up 52 different phenotypes affecting skull, teeth, ribs, spine, pelvis and limbs. This represents a monumental task, generating some 166,000 annotated images from 34,000 animals to date. However, this body of data only represents 7,500 of the 20,000 genes phenotyped so far by the IMPC. The bottleneck of manual annotation is a daunting prospect for the project and represents an unmet need for automated image analysis methods within the life sciences community.

In recent years, convolutional neural networks (CNNs) have risen to prominence for their seemingly universal applicability to a wide range of image classification problems. This partly due to the fact they require little prior domain knowledge to implement, and the data require minimal pre-processing order to achieve state-of-the-art performance. However, there are nevertheless challenges that preclude their adoption for large-scale phenotyping. Traditional CNNs are not naturally suited to datasets where the number of input images is variable; an individual animal may be captured from one and up six different viewpoints in practice. Furthermore, CNNs trained to perform multiple tasks at once (i.e., one animal may exhibit multiple phenotypes) have little appreciation or knowledge of the relationships between the tasks due to anatomical proximity. Beyond the immediate use-case of CNNs for automation, there is an opportunity to leverage the internal representations learned by CNNs to perform large-scale data mining and support biological discovery.

HiDRA (Hierarchical Deep Representations of Anatomy) will address these challenges by developing a "multi-view-multi-task" approach that is robust to variations in the input data, shares information between anatomically-related tasks, and learns anatomy-specific feature representations for individual animals. Information fusion from multiple viewpoints will allow for any number of images to be provided and help to indicate which view was most informative for annotation. To account for the relative scarcity of individual phenotypes, a hierarchical training scheme will be developed to share information across related tasks according to anatomical proximity. The learned representations will also be subject to constraints that minimise correlations between anatomical structures, allowing for comparisons to be made between animals in an anatomy-specific fashion using data mining techniques. Among the outputs of this research will include computational tools made available to the wider life sciences community for analysis of x-ray data at any scale.