Super Resolution with Deep Learning for Image Recognition

Deep learning is finding several applications in the automated recognition and identification of objects in large datasets. A particularly active research area is its application to super resolution in biomedical imaging. A typical example is the ability to produce high resolution images of cell structure by training a machine to correct blurred images by adapting image processing protocols to optimise the recovered image against a corresponding high quality image. Having "learned" the processing strategy with a wide variety of cell images and optical conditions, a blind test can then be performed on a blurred image for which there is no corresponding high quality image. The machine is able to "analyse", adapt, optimise and apply a processing strategy to recover a high quality image based on what it has previously learned using training images which are similar to the object under test.
The applicants and Leonardo are interested in understanding how the same methodology can be applied to object recognition in an airborne of land-based scenario. The problem is characterised by several practical considerations:
- There is more than one type of object to detect.
- The background terrain is varied.
- The camera (2D or 3D) provides a limited spatial resolution determined by the lens system (dependant on optical aberrations) and the sampling resolution (determined by the pixel resolution and pitch of the focal plane array). These determine the angular resolution of the camera. For 3D imaging the range resolution should be considered. The question then is: can the image be improved to approach the diffraction limit?
- The object will usually need to be tracked and therefore the output should be at video rates. This also implies that the background is also variable during the track.
- At long range atmospheric turbulence will distort the image in a varying manner. This means that the blurred image is continuously changing even for a stationary object.

PROJECT AIM
The aim of the study is two-fold:
1. Super Resolution of Optical Systems. Determine the processing protocols necessary for a deep learning approach to object recognition. One approach is to adopt a similar computational scheme as that described in [1] for super resolution imaging of cells but changing the target set, tags and training database to fit a terrain-based, object recognition problem. For example, a database comprising high angular resolution images of vehicles taken at short range might be used to train a machine to recognise a vehicle in a class of vehicles whose images are taken at long range, where the finer detail is not well resolved and limits the use of shape and contour recognition.
2. Image Recognition and Identification Through Turbulence. In this application the principle is very similar to that described in [1]. A training database comprising blurred images of objects (taken through turbulence) and the corresponding blur-free image (taken through very low level turbulence) is used to train a machine to recognise a blurred image . The protocol is then used on a blurred image of an object not included in the database to conduct a blind test. The difference between this and [1] is that the blurring will change over time and therefore several blurred images will be associated with a single blur-free image. The key questions are: what level of turbulence can be tolerated, to what extent does light level affect the performance (e.g. dusk and dawn) and what is the processing speed.

REFERENCES
[11] Y. Rivenson, Z. Goroc, H. Gunaydin, Y. Zhang, H. Wang, and A. Ozcan, "Deep Learning Microscopy", Optica, vol. 4, no. 11, 1437-1443, 2017.