Programmable embedded platforms for remote and compute intensive image processing applications

Image processing is playing an increasingly important role in our lives whether this is the numerous sources of social provision e.g. TV, or the increased reliance on security to protect our everyday lives through the proliferation of security cameras in airports and town centres. There are also healthcare applications with increased need for 3-dimensional (3D) images such as in viewing 3D computerised tomography scans to provide much more intelligent treatment. In automotive applications, cameras are used for quality assurance in manufacture and situational awareness in use. In security applications, organisations are keen to have more intelligent views of scenes to highlight security risks and dangers. This has increased the amount of visual information that we process and store, and has placed increasing importance on the users' ability to process data where it is received, thus pushing for more intelligent image processing.
Whilst a lot of innovative work has been done to derive the algorithms to provide this intelligence, there is a clear need for suitable, high performance, lower power hardware to provide the processing as in many cases, these systems may be remote e.g. security cameras with limited interconnection. We could wait for technology evolutions to provide the increased performance as before, but the warnings on process variability below 45-nm CMOS technology suggest that this might not be forthcoming and implies an increased focus on novel processor architectures is required. Whilst multi-core and application specific processors such as graphical processing units (GPUs) have been proposed, the gains have been limited. In addition, the rapid developments in the acquisition and interpretation of images together with intelligent algorithmic development, have not been matched by sound software engineering principles to develop and transform code into hardware implementations efficient in speed, memory and power. In many cases, image sensors comprise simple processing engines which communicate to some central resource for further processing. For a lot of medical and security applications, there is a need for more intelligent image acquisition, multi-view video processing (merging many views into a more useful, higher-level representation) and more context-aware acquisition devices which are aware of the existence of other cameras which can contribute to the creation of the full scene. This requires a step change in how we design and program these systems.
Current FPGA technology such as the Xilinx Virtex-7 FPGA, offers a huge performance capability (over 6.7 Giga Multiply-Accumulate per second and up to 30 Terabits/s of memory bandwidth) and better power efficiency than GPUs. Currently FPGA solutions are created by aggregating powerful intellectual property (IP) cores together with soft cores, but the resulting performance is limited by the overall systems architecture and programmability is severely limited. Hence, there is a clear need to derive a FPGA system architecture that best matches the algorithmic requirements but that is programmable in software for a range of algorithms in the application domain. By considering the model of computation and programming model from the outset, we propose to create a highly powerful platform for a range of image processing algorithms. The proposal combines the FPGA processor design expertise in Queen's University (Woods), with the software language and compiler research (Michaelson) and image processing expertise (Wallace) at Heriot-Watt University. A key aspect is to ensure close interaction between the processor development and software languages and representation, in order to ensure the creation of a processor architecture configuration that is programmable in software. The research looks to radically alter the design of front end image processing systems by offering the performance of FPGA solutions with the programmability of processor solution

What Benefits Will Accrue from This Research?
The ability to implement highly complex image processing algorithms on an FPGA-based processor architecture will change how system designer implement security applications. It will now allow clever image processing to be undertaken at source and thus alter the dynamics of how image processing systems are developed. For medical applications, it presents the ability to process data again at source and should accelerate the development of new types of algorithms for performing medical analysis. This should result in improved image processing systems and major implications for biomedical and security applications.

What are The Economic Benefits of this Research?
The target area for this research is to create an image processing solution that can be used in the medical, automotive/aerospace and security markets. These markets have considerable commercial opportunity. For example, the global market for imaging products in the medical domain is $28B (2007 TriMark) annually with a clear need move from detection to prevention and complexity such as combining multiple images in 3D imaging. The automotive/aerospace machine vision for manufacture is driven by the need to create new inspection systems utilizing multiple image sources under a variety of lighting conditions is worth £2.7B per year in Europe (410M in the UK). The security market is dominated by surveillance, accounting for around 50% of the total £12B global market (2010 Memoori) with Europe growing from £490M (2009) to £610M (2015). The applicants have a proven track record in commercialising research and the engagement with, Andor, CapnaDSP, Thales and Xilinx will ensure a direct route to commercialisation.

Ensuring Economic Impact
Woods is a co-founder of CapnaDSP Ltd., a University spinout that is looking to commercialising recent FPGA design tool developments in the group. As the letter of support indicates, CapnaDSP are very interested in novel FPGA-solutions and are beginning to gain traction in the area of the image processing solutions; they would be keen to ensure relevance of the resulting technology to their current design problems and would be keen to potentially license the resulting technology. The notion of applying FPGA technology to image processing is of considerable interest to Andor and Xilinx (see letters of support). Xilinx will be keen to interact with the project team and will supply equipment and software to support this project; we plan for visits to both the Dublin and San Jose offices to ensure useful engagement. In addition, the PI will also use a long-standing involvement with E-Futures and the UK Design Forum to inform the UK electronics community. Wallace (HWU) has existing collaborative programmes with both Thales and Selex Galileo on mobile vehicle awareness and airborne sensing respectively. Each is building current prototypes for future products based on these technologies, and in particular we include explicit collaboration with Thales in this proposal.

What are The Skills/Trained People Benefits of this Research?
The researchers on this project will gain from the highly commercial focus at both ECIT and the University to develop a good balance technical expertise developed in close conjunction with industry. Furthermore, the opportunity to engage directly with Andor, Xilinx and others will expose the projects' researchers to world class image processing expertise. Research Fellows and PhD graduates from previous EPSRC funded research programmes are involved with ARM, CSR, Nuance and SAP as well as a number of spin-off companies.