Detailed malaria diagnostics with intelligent microscopy

Lead Research Organisation: University of Bath
Department Name: Physics

Abstract

The best way to diagnose malaria remains microscopic examination of blood smears, to identify the plasmodium parasites that are responsible. This takes around 30 minutes of microscopy, done by a trained technician - skilled workers who are in short supply. This project will create an intelligent microscope that can greatly multiply the skills of a technician by scanning over the smears automatically, and allowing them to review only the suspicious blood cells on a tablet computer after the smear has been scanned.

Malaria is one of the world's most prevalent infectious diseases. It affects 200 million per year, and causes around 400 thousand deaths - most of them children in ODA countries in sub-Saharan Africa. Impressive progress is being made in reducing the incidence of malaria, which makes good diagnosis of the condition ever more important; it is increasingly inaccurate to assume that every patient with a fever has malaria, and doing so will waste drugs and leave potentially life threatening fevers untreated.

The key to reliable, useful diagnosis with an automated microscope lies in computer vision; simply acquiring digital images and tiling them together into a digital smear is an important first step, but robust analysis of the digital images means the technician need not sift through many images of healthy cells. Instead, they can concentrate their efforts on parts of the image where the algorithm identified suspicious features. Once trained, our algorithm will be able to identify many parasites, only asking for the technician's opinion in challenging, ambiguous cases when it could not identify objects with certainty. Fully automated counts of healthy and infected cells will then allow consistent quantification of test results, informing the clinician prescribing treatment and aiding in disease monitoring.

Analysis of medical images raises fundamental issues with the standard "deep learning" approach of training a multi-layer neural network on hundreds of thousands of images. Such algorithms cannot accurately quantify their uncertainty (i.e. flag up when a diagnosis may be inaccurate), nor describe the reasoning that led to a given classification for an image. They require extremely large training datasets, which must often be labelled by hand. We will build a generative probabilistic model which, while not feasible in most applications due to the huge range of objects that might conceivably be found in a photograph, is possible in the relatively controlled imaging environment of a microscope. This will allow us to give a probabilistic verdict on each cell, and highlight cells that couldn't be reliably classified as healthy, infected, or something else. The generative model will also be able to identify features that led to a classification, for example highlighting infected cells in a large image of a smear. Both of these features will enable greater trust in the algorithm, and allow it to be used to support, rather than replace, existing clinical staff as well as collecting images that will allow us to improve the algorithm's performance.

Computer vision is a powerful technique, but it requires high-resolution digital representations of blood smears in order to work. Our project therefore has a hardware component, where we will build on our earlier work with the OpenFlexure Microscope to create a slide-scanning instrument, capable of digitising blood smears in the field. This instrument will use low cost components and desktop digital manufacturing, so that it can be produced locally - freeing clinics from expensive international supply chains, and creating opportunities for local entrepreneurs that build valuable engineering and design skills. We have already trialled this approach with the first version of the microscope, which will shortly be available for purchase in Tanzania and Kenya, and we hope to achieve an even greater impact with a fully automated instrument.

Planned Impact

The ultimate beneficiaries of this work will be patients in sub-Saharan Africa, who will have faster, more reliable malaria diagnosis available to them and thus receive better healthcare - particularly as progress is made in lowering the incidence of the disease. This benefit will come through healthcare professionals, particularly technicians, who are able to treat and diagnose more patients thanks to assistance in blood smear analysis from the software and hardware instrument we will develop in this project. By replacing time-consuming manual microscopy with automated microscopy, and subsequent review of samples on a screen, we will not only multiply the skills of trained technicians, but make it much easier for trainee technicians to learn the features to look for when analysing a blood smear. Sharing a screen is far easier than observing the same sample through microscope eyepieces, and images displayed on-screen are much easier to compare with reference images than microscope slides viewed through eyepieces.

By working with Ifakara Health Institute, a Tanzanian research organisation, we ensure that our research can be immediately used to help certify a medical device in Tanzania; IHI have an ongoing relationship with the national standards bodies NIMR and TFDA and are familiar with the approval process. We have also carefully considered how our improved method will gain acceptance by medical technicians - by improving their existing practice rather than replacing it with a totally new method, or replacing the technician with a machine, we enhance their jobs rather than remove them. The instrument will be able to explain its results, highlighting parasites, eliminating healthy cells, and flagging areas where it cannot classify a feature. These difficult areas can be reviewed by a trained technician, ensuring the instrument does not give unreliable diagnoses and allowing us to collect useful images to improve future versions of the algorithm.

Our strategy of creating locally manufacturable instruments will also benefit local engineers, technicians and entrepreneurs. Releasing our designs open source, and making extensive use of digital manufacturing techniques such as 3D printing, enables high quality automated microscopes to be brought to market in ODA nations without relying on expensive imports, or the legal and financial complications resulting from patents. By moving as much as possible of the commercialisation process to Tanzania, we can bring a product to market faster and with less capital. We also ensure that local entrepreneurs adapt and market the instrument to meet local needs, and that there is an after-sale support network that does not rely on expensive international shipping. Through the Tech for Trade network, we will be able to quickly grow the number of countries where this technology is produced, and we have included budget for some initial work in Kenya.

We are also committed to public engagement in the UK, and our use of low cost hardware lends itself to workshops and exhibitions from school age to adults. We will attend science festivals, such as the Bath Taps into Science festival, as well as working with other groups to run workshops around our open source designs. We are also working with the Raspberry Pi foundation to create teaching materials related to computer-controlled microscopy, joining up ICT, Physics and Biology and bringing school lab technology up to date with modern developments.
 
Title Microscope Folk comic 
Description As well as being exhibited at the "visions of science" exhibition, Tom Armstrong's comic book about a visit to our lab, "the microscope folk" has been distributed in print and online to many interested parties, collaborators, and colleagues. 
Type Of Art Artwork 
Year Produced 2018 
Impact This output was exhibited at an exhibition on campus, and has also provided imagery that we have used to illustrate talks - particularly to non scientific audiences, where the comic book style helps to make our content more approachable. 
 
Description We have developed and released a hardware and software design for a digital microscope that is capable of taking the high quality images required for malaria diagnostics. The impact of this is on healthcare (SDG3) and education (SDG4) in Tanzania, an ODA country. By sharing our designs as Open Source Hardware, we are able to spread this impact across the developing world and are actively working to do so with our partners Bongo Tech & Research Labs (a Tanzanian company, currently working towards medical certification for the microscope). We are also working with groups around the world to bring our designs to a wider audience - including a manufacturer in Germany and academic and community labs across Africa and South America. Our recent survey showed 40 countries where people had made or used OpenFlexure microscopes.

We have recently made an alpha release of Version 7, focused on reliability and manufacturability, which marks a step change in the quality of the design-for-manufacture and the build instructions. This is key to the OpenFlexure microscope's popularity, and is a major reason our project has been picked up by so many groups. Our software has broken new ground by using standard Internet of Things libraries to control scientific hardware: this has wide-reaching potential for making laboratories more efficient and more automated, and it has already been adapted for use in other systems.

In the final year of the award, we have made significant progress with machine learning for analysis of the captured images. This prioritises "explainability", meaning that the algorithm can not only communicate whether it has detected parasites, but can show where and why they have been detected and give an estimate of its confidence. We believe this work is crucially important to inform the use of machine learning in healthcare.
Exploitation Route Our hardware designs are already being reproduced by many people around the world, as documented on the website, online forum, and gitlab repositories. This allows local manufacturers in ODA countries such as Tanzania, Kenya and Ghana to produce high tech equipment (SDG9) that is used for healthcare (SDG3) and education (SDG4). We are now pursuing medical certification for the microscope, and our ambition is for the OpenFlexure Microscope to be manufactured by STICLab in Tanzania, the first in-vitro diagnostic device to be made domestically.

Several groups around the world (including potential manufacturers in North and South America, Africa, and Europe) have already expressed interest in manufacturing the open source hardware design, and we have applied for IAA funds to support this tech transfer.

Many academic and community labs are working with our microscope, doing everything from cutting-edge microscopy technique development to citizen science for pollution monitoring, and we fully expect this to continue in the future.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Healthcare,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology

URL https://www.openflexure.org/
 
Description We have worked wth local producers Bongo Tech & Research Labs (BTech, formerly STICLab) in Tanzania to demonstrate that our microscope design can be locally produced to a standard that is capable of serious clinical imaging. These microscopes are now routinely produced by BTech and they are pursuing them as a commercial project, for marketing to schools. We are also seriously investigating medical certification for BTech and for our hardware and software. So far, BTech has registered as a medical device manufacturer, passed its premises inspection, and undertaken ISO13485 training. If successful, locally manufactured microscopes would be a major impact on the healthcare system in Tanzania, and a major economic step forward with the first locally-manufactured in-vitro diagnostic device. The primary ODA country benefitting from this impact is Tanzania, though as we have shared all our software and designs as open source, we expect more countries will follow suit - particularly Ghana, where we have strong connections through the Tech for Trade network and Kumasi Hive. This work contributes strongly to SDG9, through developing the local high tech manufacturing capability at STICLab and more generally through "maker" spaces equipped with locally producible 3D printers. Our findings are aimed at addressing SDG3 (healthcare) through improving malaria diagnosis, and also SDG4 (education) by providing quality science teaching apparatus. As well as our close collaboration with BTech in Tanzania, we are working with a German manufacturing company to provide kits for sale worldwide; we have spun out a UK start-up to provide kits domestically, but this is not yet able to ship worldwide. We are in discussion with potential suppliers in the USA and South America. The OpenFlexure Project has really exemplified the potential of open source hardware, as our vibrant community includes a huge range of groups - from professional scientists in the UK, through to community groups using it for air quality monitoring around the world. Some examples include Daniel Rosen, a pathologist at Baylor College of Medicine, USA, who has used the OpenFlexure Microscope to digitise biopsy specimens taken in labs in South America and Africa, where slide-scanning facilities are otherwise unavailable, or a community group in Argentina using OpenFlexure microscopes to examine soil microbiology as part of a research project. The huge strength of our open approach is that these groups are not simply using outputs from our project, but there is a meaningful two-way engagement through our online community forum. This in turn informs future development directions of the project, and leads to volunteers contributing improvements. A recent survey had responses from 40 countries where people had made or used the microscope. As reproducibility and openness in experimental science become more strongly emphasised, our project is leading the way towards better sharing of experimental details. The popularity of what we have created makes us an excellent testbed for issues around licensing, commercialisation, and negotiating with a University system that is habituated towards the traditional patent-and-license model. OpenFlexure has been selected as a case study, and our own reflections on sharing designs have been picked up by UNESCO, the Wilson Centre, and the Engleberg Centre in their discussions of these issues. We were also called upon by several groups responding to the COVID-19 pandemic, because of our expertise in sharing hardware openly.
First Year Of Impact 2020
Sector Education,Electronics,Healthcare,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology
Impact Types Societal,Economic

 
Description IAA - The OpenFlexure Diagnostic Microscope
Amount £44,037 (GBP)
Organisation University of Bath 
Sector Academic/University
Country United Kingdom
Start 01/2021 
End 12/2021
 
Description Locally manufactured smart diagnostic microscopes
Amount £15,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 03/2019 
End 01/2020
 
Description URF Enhancement Award
Amount £200,000 (GBP)
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2018 
End 03/2021
 
Description University Research Fellowship
Amount £600,000 (GBP)
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2018 
End 09/2023
 
Title OpenFlexure Microscope v5.20 
Description The OpenFlexure Microscope is a 3D printable design for a microscope, equipped with a high quality motorised stage for automatic focus and sample translation. It includes a range of options for the optics, from a very simple webcam-based design (suitable for use in schools, for example) to a lab-ready design with condenser optics on the transmission illumination, and a conventional microscope objective for high resolution imaging. As well as being inexpensive to produce, this microscope is designed parametrically, which makes it simple to customise for different use cases. By integrating all of the control electronics, including an embedded Raspberry Pi computer, into a portable, low cost, low power device, we have created a research tool that will enable long-running experiments, or multiple experiments in parallel, without the need to tie up expensive microscopy equipment that is often in high demand by multiple users. These same advantages make our tool particularly appropriate for healthcare and research scenarios in the developing world, or for use in education scenarios where budgets are limited. NB the date below relates to the most recent "release" of the project which represents a step change in performance and functionality. An earlier version of the microscope has been in circulation since 2015. 
Type Of Material Technology assay or reagent 
Year Produced 2018 
Provided To Others? Yes  
Impact Our microscope design is in use across the world, in countries including Kenya, Tanzania, Ghana, Paraguay, and Chile - mostly in research labs and schools. It has also been reproduced in research labs in the UK, Germany, the US, and more for use in research projects. 
URL https://github.com/rwb27/openflexure_microscope/
 
Title Raspberry Pi camera lens shading code 
Description The Raspberry Pi camera module is a popular image sensor in many open source scientific hardware projects due to its low cost and high performance. One major limitation is in the firmware that applies a "lens shading correction" to images, which is problematic when using the camera with custom optics. We have improved an open source library, to allow the use of custom calibrations and manual gain settings with the camera. 
Type Of Material Technology assay or reagent 
Year Produced 2018 
Provided To Others? Yes  
Impact Our improvements have been acknowledged by the original authors of the libraries and will be incorporated "upstream". 
URL https://github.com/rwb27/picamera/releases
 
Title Dataset for "Flat-field and colour correction for the Raspberry Pi camera module" 
Description This repository contains the hardware (OpenSCAD/STL files) and build instructions, software (Python scripts and Arduino firmware), data analysis (iPython notebook), and manuscript describing how to calibrate the colour response of a Raspberry Pi camera module. It also includes the calibration images acquired during the preparation of the work. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
URL https://researchdata.bath.ac.uk/id/eprint/764
 
Title Dataset for "Robotic microscopy for everyone: the OpenFlexure Microscope" 
Description This dataset contains microscopy images collected to demonstrate imaging capabilities of the OpenFlexure Microscope. Images for bright-field transmission and reflection, polarisation contrast, and fluorescence imaging are provided. A set of images obtained from a large tile scan are provided, along with the Microsoft Image Composite Editor file used for tiling. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
URL https://researchdata.bath.ac.uk/id/eprint/734
 
Title Chief Ray Angle Compensation for the Raspberry Pi Camera Module 
Description This repository contains the hardware (OpenSCAD/STL files and build instructions, software (Python scripts and Arduino firmware), data analysis (iPython notebook), and manuscript describing how to calibrate the colour response of a Raspberry Pi camera module. analysis contains the data analysis code. data contains the images that we used for the graphs in the manuscript. neopixel_driver is the arduino firmware. image_acquisition includes the Python code that acquired the images and controlled the neopixel. calibration_jig contains the printable files, source OpenSCAD files, and assembly instructions for the calibration jig. colour_test_sheet contains source Inkscape SVG files and PDF renders of the test target used in the experiments. manuscript contains the source files for the manuscript. This is an open project, run by the Bath Open Instrumentation Group, part of the University of Bath. Unless otherwise specified, all code is licensed under the GPL v3 or later, hardware is under the CERN open hardware license, and documentation/manuscript is CC-BY 3.0 or later. The repository will be archived along with a published paper once it has been peer reviewed. If you are viewing a static archive of these files, you may want to consult the working repository, which may receive updates in the future. 
Type Of Technology Software 
Year Produced 2020 
URL https://zenodo.org/record/3699730
 
Title Chief Ray Angle Compensation for the Raspberry Pi Camera Module 
Description This repository contains the hardware (OpenSCAD/STL files and build instructions, software (Python scripts and Arduino firmware), data analysis (iPython notebook), and manuscript describing how to calibrate the colour response of a Raspberry Pi camera module. analysis contains the data analysis code. data contains the images that we used for the graphs in the manuscript. neopixel_driver is the arduino firmware. image_acquisition includes the Python code that acquired the images and controlled the neopixel. calibration_jig contains the printable files, source OpenSCAD files, and assembly instructions for the calibration jig. colour_test_sheet contains source Inkscape SVG files and PDF renders of the test target used in the experiments. manuscript contains the source files for the manuscript. This is an open project, run by the Bath Open Instrumentation Group, part of the University of Bath. Unless otherwise specified, all code is licensed under the GPL v3 or later, hardware is under the CERN open hardware license, and documentation/manuscript is CC-BY 3.0 or later. The repository will be archived along with a published paper once it has been peer reviewed. If you are viewing a static archive of these files, you may want to consult the working repository, which may receive updates in the future. 
Type Of Technology Software 
Year Produced 2020 
URL https://zenodo.org/record/3699731
 
Title OpenFlexure eV 
Description OpenFlexure eV is a modern, Electron-based application that allows interactive control of the OpenFlexure Microscope. It was developed by our team to enable a wider range of users to work with the microscope. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact The software is in use by a number of groups around the world, in Tanzania, Kenya, Ghana, Cameroon, the USA, Germany, Chile, Peru, Paraguay, and many more. The software enables our work at Ifakara Health Institute where we are trialling the microscope for malaria diagnostics. 
URL https://gitlab.com/openflexure/openflexure-microscope-jsclient
 
Company Name OPENFLEXURE INDUSTRIES LTD 
Description OpenFlexure Industries supplies kits of open-source hardware projects to enable a greater number of people to engage with the project. 
Year Established 2019 
Impact As a recently-established microbusiness, OpenFlexure Industries has only just started selling products, but has been chosen as a case study by the Gathering for Open Science Hardware to study how sustainable businesses can be built on open hardware.
Website https://openflexure.com
 
Description Article in The Conversation 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Commissioned to write an article for The Conversation on open hardware, and how it might help to improve medical supply chains in the future.
Year(s) Of Engagement Activity 2021
URL https://theconversation.com/making-hardware-open-source-can-help-us-fight-future-pandemics-heres-how...
 
Description Exeter Festival of Physics workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact 15 school, pupils and members of the public attended a workshop where we build and then used 3D printed microscopes to view a number of samples, and explore physical computing concepts by writing an autofocus routine in Python.
Year(s) Of Engagement Activity 2018