Classification of oral lesions using deep learning for early detection of oral cancer

Lead Research Organisation: Kingston University
Department Name: Faculty: Science Engineering & Computing

Abstract

For the majority of cancers, early detection results in better survival. Oral cancer is one of the few cancers that is visible and many of these cancers are preceded by a potentially malignant lesion where medical intervention can prevent the development of cancer. Taken together, oral cancer presents an opportunity for early detection. However, identifying which oral lesion has a propensity to become oral cancer is not straightforward without specialised training and this problem is confounded by the lack of specialists who are trained in this expertise particularly in low- and middle-income countries, where the majority of oral cancers are diagnosed. One innovative approach to overcome this is to develop an artificial intelligence algorithm to classify oral lesions into those that are benign and those that are potentially malignant or are occult cancer so that patients can be triaged accordingly to receive appropriate clinical management. In this project, we propose to work within a multi-disciplinary, international team to collate a library of images from existing and prospective collections that will facilitate the development of an artificial intelligence algorithm that will be tested and validated. The outcome of this project will pave the way for further rigorous testing, development of an App incorporating this automated tool and clinical validation for the early detection of oral cancer. The development of an automated tool for the classification of oral lesions will facilitate the identification of patients most at risk to develop oral cancer so that these individuals can be managed appropriately. This project is particularly impactful in the low- and middle-income countries as the majority of the global burden of oral cancer is found in these countries.

Technical Summary

About 300,000 individuals are diagnosed with oral cancer every year and the majority of these patients live in low- and middle-income countries. Up to 80% of oral cancers are preceded by early lesions called oral potentially malignant lesions (OPMD) and this presents an opportunity to identify and manage these lesions to reduce the chances of malignant transformation. However, due to the numerous types of lesions that present in the oral cavity, the identification of OPMD is not straightforward. Further, this is confounded by the limited number of trained oral medicine specialists in countries where the prevalence is at its highest.

One way to overcome the lack of access to expertise for the clinical diagnosis of these lesions is to use deep learning approaches to develop an algorithm that could automatically classify images into different categories that require distinct clinical management. Such an approach has been successfully used for the classification of skin diseases. In this project, we will work within a multi-disciplinary team to use deep learning for the classification of oral lesions. In particular, we will employ two different deep learning architectures - VGG-16 and ResNet, pre-trained using ImageNet, and apply transfer learning to classify oral lesions.

The outcome of this project will pave the way for further rigorous testing, development of an App incorporating this automated tool and clinical validation for the early detection of oral cancer. The development of an automated tool for the classification of oral lesions would facilitate the identification of patients most at risk of developing oral cancer so that these individuals can be managed appropriately.

Planned Impact

This project has benefits to different stakeholders:
Clinicians and Ministries of Health: An automated tool that could help with patient triaging to streamline healthcare would be very useful for primary healthcare practitioners to manage their patients. This would also ensure that specialists are seeing patients that are most at risk, streamlining resources that are often limited in low-and middle-income countries. As oral cancer is very often treated in the public hospitals, such an automated classification tool would have the farthest reach through the Ministry of Health. In Malaysia, we are already partnering with the Ministry of Health to conduct clinical trials on MeMoSA and the same network will be involved in this project. This strong and long-lasting partnership will facilitate the testing and use of the automated tool in the longer term and together with other research, including health economics, data from this project may play a role in shaping healthcare policies in Malaysia and other countries in the region.

Communities in low- and middle-income countries and globally: An automated tool for early detection of oral cancer would be most beneficial to low- and middle-income countries where there are not enough specialists to diagnose patients. We have already demonstrated that a mobile phone App can be used to facilitate early detection of potentially malignant disorders in the oral cavity, and internet connectivity in low- and middle-income countries permits the use of telemedicine. Taken together, combining these (automated classification and telemedicine) would be an innovative and cost-effective approach to address the challenges faced by low- and middle-income countries.

Commercial partners/Social enterprise: To facilitate the wide use of a successfully developed tool for early detection of oral cancer would require a commercial partner. Therefore, in the longer-term, discussions with commercial partners would have to take place or team members may decide to set up a social enterprise so that funds can continue to finance further projects. Intellectual property will be co-owned by the investigators of this project. Currently such a tool is not available in the market and therefore there is a large potential for commercialization.

Academic/Research communities: This project will be one of the first to provide evidence that artificial intelligence approaches can be used for the classification of oral lesions and will stimulate the field for further development of deep neural networks for other diseases involving medical imaging, and could inspire the development of multi-disciplinary teams to address healthcare issues. Further, this research would also expand other areas of research including health economics to determine the cost-savings as a result of implementing an automated classification tool for the early detection of oral cancer. Data and results from this study will be disseminated at scientific conferences and through publication.
 
Description The goal of this pump-priming project has been to explore the potential of deep learning to identify high-risk precancerous and cancerous lesions in the oral cavity. This research could potentially lead to better low-cost diagnosis of a disease that is prevalent in low- and middle-income countries. The approach has included both the collection of data and the development of classification algorithms.

An important contribution to this overall goal has been the development of a library of annotated lesions that represent the different classifications of oral cancer according to the World Health Organisation. Images and metadata were collected from collaborators across five countries. Approximately 10,000 images were collected with accompanying ground truths. A subset of 2000 images were also annotated to identify and describe individual lesions. An annotation tool has been developed and a team of international clinicians have been trained and supported in its use. A maximum of seven and a minimum of three clinicians annotated each lesion. The annotations have been analysed for concordance of clinical opinion across clinicians and have contributed to the evaluation of the deep learning algorithms.

The AI algorithms developed during the course of the project have demonstrated an accuracy of more than 80% (Outputs listed in this project DOI: http://dx.doi.org/10.1007/978-3-030-64511-3_3, DOI:http://dx.doi.org/10.1109/ACCESS.2020.3010180, http://dx.doi.org/10.1007/978-3-030-64511-3_3, http://dx.doi.org/10.1109/ACCESS.2020.3010180) with potential for further improvement using larger datasets for training. This algorithm performance demonstrates the potential of the techniques to identify high-risk precancerous and cancerous lesions in the oral cavity. Both image classification and object detection approaches were explored. The work has enabled the team to explore and apply attention mechanisms within deep learning algorithms that have enhanced detection performance. This approach has relied upon the rich annotations of the images provided by the clinical teams enabled by the customized annotation tool developed during this project. The incorporation of attention mechanisms will be important to enable the AI algorithm to indicate to users which areas of the image the AI algorithm is basing its assessment on, therefore increasing confidence in the use of the algorithm among the medical community.

Overall, the outcomes from this work have provided confidence in terms of the initial performance of the deep learning algorithms to anticipate that, given a large enough dataset to train on and further research, a high-performance system could be realised that will enable the system to be implemented reliably in a clinical setting. With respect to ODA compliance, an effective international, interdisciplinary team with active communication channels was established during the project. Future work will build on this established network, that represents both technical and clinical expertise, and will provide a sound basis for the inclusion of other clinical partners planned to build a dataset of significant size for the deep learning development.
Exploitation Route The outcomes of this work demonstrate the potential of deep learning algorithms to identify high-risk precancerous and cancerous lesions in the oral-cavity. The reported work could lead to further development of deep learning algorithms to achieve a performance where the algorithms can be clinically deployed.
Sectors Healthcare

 
Description Ethnic differences in performance and perceptions of Artificial Intelligence retinal image analysis systems for the detection of diabetic retinopathy in the NHS Diabetic Eye Screening Programme
Amount £497,157 (GBP)
Funding ID AI_HI200008 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 10/2021 
End 10/2023
 
Description Prediction of complications of diabetes mellitus utilising novel retinal image analysis, genetics, and linked electronic health records data
Amount £1,126,103 (GBP)
Funding ID 224390/Z/21/Z 
Organisation Wellcome Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 04/2022 
End 04/2025
 
Description Clinical collaborations 
Organisation JSS Dental College and Hospital
Country India 
Sector Academic/University 
PI Contribution The research team updated the clinical collaborators (New York University, US; Trisakti University, Indonesia; and JSS Dental College and Hospital, India) on project progress and enabled clinicians to use software produced specifically to annotate images of oral lesions. Feedback was provided to clinical collaborators regarding annotation concordance between clinical partners and overall results of the project were circulated.
Collaborator Contribution Clinical partners supplied expertise in classification of oral lesions and time regarding collection and annotation of oral lesion images.
Impact The clinical partnerships contributed to all publication outputs listed in this project on Research Fish: DOI: http://dx.doi.org/10.1007/978-3-030-64511-3_3 DOI:http://dx.doi.org/10.1109/ACCESS.2020.3010180 The collaboration is multi-disciplinary, enabling the technical, clinical and scientific team included on the original application to interact with these extra clinical partners.
Start Year 2019
 
Description Clinical collaborations 
Organisation New York University
Country United States 
Sector Academic/University 
PI Contribution The research team updated the clinical collaborators (New York University, US; Trisakti University, Indonesia; and JSS Dental College and Hospital, India) on project progress and enabled clinicians to use software produced specifically to annotate images of oral lesions. Feedback was provided to clinical collaborators regarding annotation concordance between clinical partners and overall results of the project were circulated.
Collaborator Contribution Clinical partners supplied expertise in classification of oral lesions and time regarding collection and annotation of oral lesion images.
Impact The clinical partnerships contributed to all publication outputs listed in this project on Research Fish: DOI: http://dx.doi.org/10.1007/978-3-030-64511-3_3 DOI:http://dx.doi.org/10.1109/ACCESS.2020.3010180 The collaboration is multi-disciplinary, enabling the technical, clinical and scientific team included on the original application to interact with these extra clinical partners.
Start Year 2019
 
Description Clinical collaborations 
Organisation Trisakti University
Country Indonesia 
Sector Academic/University 
PI Contribution The research team updated the clinical collaborators (New York University, US; Trisakti University, Indonesia; and JSS Dental College and Hospital, India) on project progress and enabled clinicians to use software produced specifically to annotate images of oral lesions. Feedback was provided to clinical collaborators regarding annotation concordance between clinical partners and overall results of the project were circulated.
Collaborator Contribution Clinical partners supplied expertise in classification of oral lesions and time regarding collection and annotation of oral lesion images.
Impact The clinical partnerships contributed to all publication outputs listed in this project on Research Fish: DOI: http://dx.doi.org/10.1007/978-3-030-64511-3_3 DOI:http://dx.doi.org/10.1109/ACCESS.2020.3010180 The collaboration is multi-disciplinary, enabling the technical, clinical and scientific team included on the original application to interact with these extra clinical partners.
Start Year 2019