Non-Imaging optics Makes for Richer Observation Data
Lead Research Organisation:
University of Surrey
Department Name: Vision Speech and Signal Proc CVSSP
Abstract
NIMROD: Non-Imaging optics Makes for Richer Observation Data These new sensors will use Imaging ?
to create a new type of single pixel camera. Although these techniques are common in the solar domain, this is first time
anyone has attempted to use signal processing techniques to build an imaging system out of non-imaging optics. The
potential of the NIMROD attentional sensors is extraordinary, but there are significant research challenges to be overcome
in order to realise this step-change. The visual data produced by these sensors is utterly unlike that of a traditional
camera, rendering decades of research on computer vision and artificial intelligence unusable. This studentship will focus
on developing the fundamental signal processing, imaging and machine learning techniques necessary to extract meaning
from such a data stream.
to create a new type of single pixel camera. Although these techniques are common in the solar domain, this is first time
anyone has attempted to use signal processing techniques to build an imaging system out of non-imaging optics. The
potential of the NIMROD attentional sensors is extraordinary, but there are significant research challenges to be overcome
in order to realise this step-change. The visual data produced by these sensors is utterly unlike that of a traditional
camera, rendering decades of research on computer vision and artificial intelligence unusable. This studentship will focus
on developing the fundamental signal processing, imaging and machine learning techniques necessary to extract meaning
from such a data stream.
Organisations
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509772/1 | 01/10/2016 | 30/09/2021 | |||
| 2115443 | Studentship | EP/N509772/1 | 01/10/2018 | 30/09/2021 | YUSUF DUMAN |
| Description | A scanning pixel camera is a low-cost, low-power sensor that is not diffraction limited. It produces data as a sequence of samples extracted from various parts of the scene during the course of a scan. It can provide very detailed images at the expense of samplerates and slow image acquisition time. We proposes a new algorithm which allows the sensor to adapt the samplerate over the course of this sequence. This makes it possible to overcome some of these limitations by minimising the bandwidth and time required to image and transmit a scene, while maintaining image quality. We examine applications to image classification and semantic segmentation and are able to achieve similar results compared to a fully sampled input, while using 80% fewer samples. As scenes will rarely be static, we have adapted our algorithm to be used with video based computer vision tasks. Modern deep-learning based methods that operate on videos are often computationally intensive, as they have to operate on all pixels of all frames of the video. We have created an algorithm to minimise redundant computations. This is done by predicting what areas of the next frame are likely to be informative based on our observation of the current frame. The resulting method can easily be applied to any sequential learning problem, making it possible to drastically reduce the bandwidth and computation requirements while maintaining consistent performance. We evaluate this for the tasks of tracking and pose estimation, and are able to reduce the data processing by 80% while maintaining 96% relative performance. |
| Exploitation Route | Further development of the hardware platform would allow the generation of real world data rather than simulating it via existing video/image data. This would likely allow for further refinement of the algorithms that have been created in this project due to the nature of the data produced. Another avenue would be finding a real world task and deploying a completed system on it. |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Description | building links between Surrey and the optics manufacturer, and potentially supporting the development of cutting edge new sensor technology Built links between the University of Surrey and the optics manufacturer that is supporting this project. This can potentially support the development of cutting edge new sensor technology around which the algorithms produced by this project have been designed. |