Programmable embedded platforms for remote and compute intensive image processing applications

Image processing is playing an increasingly important role in our lives whether this is the numerous sources of social provision e.g. TV, or the increased reliance on security to protect our everyday lives through the proliferation of security cameras in airports and town centres. There are also healthcare applications with increased need for 3-dimensional (3D) images such as in viewing 3D computerised tomography scans to provide much more intelligent treatment. In automotive applications, cameras are used for quality assurance in manufacture and situational awareness in use. In security applications, organisations are keen to have more intelligent views of scenes to highlight security risks and dangers. This has increased the amount of visual information that we process and store, and has placed increasing importance on the users' ability to process data where it is received, thus pushing for more intelligent image processing.
Whilst a lot of innovative work has been done to derive the algorithms to provide this intelligence, there is a clear need for suitable, high performance, lower power hardware to provide the processing as in many cases, these systems may be remote e.g. security cameras with limited interconnection. We could wait for technology evolutions to provide the increased performance as before, but the warnings on process variability below 45-nm CMOS technology suggest that this might not be forthcoming and implies an increased focus on novel processor architectures is required. Whilst multi-core and application specific processors such as graphical processing units (GPUs) have been proposed, the gains have been limited. In addition, the rapid developments in the acquisition and interpretation of images together with intelligent algorithmic development, have not been matched by sound software engineering principles to develop and transform code into hardware implementations efficient in speed, memory and power. In many cases, image sensors comprise simple processing engines which communicate to some central resource for further processing. For a lot of medical and security applications, there is a need for more intelligent image acquisition, multi-view video processing (merging many views into a more useful, higher-level representation) and more context-aware acquisition devices which are aware of the existence of other cameras which can contribute to the creation of the full scene. This requires a step change in how we design and program these systems.
Current FPGA technology such as the Xilinx Virtex-7 FPGA, offers a huge performance capability (over 6.7 Giga Multiply-Accumulate per second and up to 30 Terabits/s of memory bandwidth) and better power efficiency than GPUs. Currently FPGA solutions are created by aggregating powerful intellectual property (IP) cores together with soft cores, but the resulting performance is limited by the overall systems architecture and programmability is severely limited. Hence, there is a clear need to derive a FPGA system architecture that best matches the algorithmic requirements but that is programmable in software for a range of algorithms in the application domain. By considering the model of computation and programming model from the outset, we propose to create a highly powerful platform for a range of image processing algorithms. The proposal combines the FPGA processor design expertise in Queen's University (Woods), with the software language and compiler research (Michaelson) and image processing expertise (Wallace) at Heriot-Watt University. A key aspect is to ensure close interaction between the processor development and software languages and representation, in order to ensure the creation of a processor architecture configuration that is programmable in software. The research looks to radically alter the design of front end image processing systems by offering the performance of FPGA solutions with the programmability of processor solution