iSee - Intelligent Vision for Grasping

Intelligent vision is a key enabler for future robotics technology. Shadow's recent development work on a disruptive universal gripper (launched at Innovate16) has identified a need for better vision to permit the automation of grasping. Building on R&D at the University of Glasgow for over 20 years, the iSee project will establish concrete robot vision benchmarks, based on commercially relevant scenes, and then develop, validate and integrate vision sensors and processing algorithms into the Smart Grasping System (SGS) to enable it to reach significant new markets in automation, logistics and service robotics. The following candidate sensors have been selected for benchmarking:

A. Low-cost time of Flight (TOF) 3D cameras (available from various companies)
B. Stereo pairs of off-the-shelf HD cameras and stereo pairs of embedded vision sensors in conjunction with CVAS's existing custom stereo-pair image matching and photogrammetry software
C. An Asus Xtion RGBD camera will serve as a benchmark reference sensor

We propose to build an integrated hand-eye system for each sensor listed above, along with appropriate lighting, and to develop complete integrated pipelines to benchmark the different combinations of capture and analysis systems on the specified scenarios. This investigation will allow us to determine the performance of 2D and 3D sensing methods in terms of quality of image capture and hand-eye performance.

We propose also to evaluate the new and highly disruptive Deep Convolutional Neural net technology (DCNNs) that has the potential to leapfrog the best algorithmic vision methods and to provide a fast, accurate and complete vision solution that meets the demands of advanced robotic grasping and manipulation. We will thus augment the evaluation with potentially very efficient and highspeed DCNN algorithms for interpreting images using potentially low-cost sensors for:

* Detecting and localising known objects and estimating their pose for grasping purposes
* Estimating depth, size and surface normals directly from single monocular images, using transfer methods
* Recovering depth from binocular and monocular camera systems using stereo matching and structure from motion (optical flow) respectively

Once trained, DCNNs can analyse images very quickly and are now becoming suitable for low-cost embedded platforms, such as smartphones. This aspect of the proposed investigation has the potential to massively simplify the sensor in terms of hardware only single cameras, or stereo pairs of cameras, are required in combination with DCCNs as the basis for a vision system which might potentially provide all of the functionality required to control the hand in a wide range of scenarios.

Benchmark results will let us develop specific camera-algorithm combinations to improve performance for the specified use cases over a number of test-evaluate-improve iterations. The core 3D sensing approaches will be integrated with the SGS, and we shall evaluate additional cameras situated off-hand providing critical ancillary visual input when objects are close to the gripper camera prior to a grasp, or during in-hand manipulation. Both camera systems will be used to acquire different views of the scene, with both systems mounted on a robot arm for interactive perception of the scene and objects contained within it.

In parallel, we will develop a showcase demonstration system at Shadow based on Shadow's current grasp planning software coupled to the 3D images captured by the benchmarked 3D vision systems. The developed vision modules will be encapsulated within the "Blocky" programming system to afford a simple and direct method for end-users to take advantage of this capability.

In conclusion, we believe that the robotics hand-eye pipelines proposed within the iSee project have the potential to play an important role in maintaining market leadership in the development of complete robotics system solutions.

The iSee project will provide a technology that can utilise in assistive and social settings that will underpin fundamental research and commercial collaborations and will deliver impacts in the Knowledge, Economy, Society, and People areas.

Knowledge
In iSee, we will ensure that the understanding of problems and solutions generated by this activity will flow in both directions between the Shadow and the CVAS group. Therefore, direct scientific impacts will occur if the benefits of the iSee's vision library lead to its adoption within the smart grasping system. Indirect impacts will occur if iSee's integrated smart sensing is adopted in other robotic scenarios outside the current scope of the project -- e.g. assistive and care roles, human-robot interaction, etc.

Robotics and autonomous systems are recognised by the UK as one of the eight grand technologies of the future of which iSee's vision and grasping will subserve as the founding robotic platform for the development and design of new robotic and autonomous technologies. In the longer term enhanced and integrated visual sensing technologies within smart grasping systems will encourage the development of new types of robots and robotic systems. Specifically, robots will be capable of working in new areas such as constrained places in manufacturing that are challenging for humans to access, and to operate in environments not suitable for industrial vision.

Economy
Robotic technologies are increasingly used in high wage economies such as the UK, and it is anticipated to be one of the drivers of the fourth industrial revolution. The technology developed in iSee will provide the industrial sector with immediate economic impact in terms of the exploitation of new product sales and profitability that is tightly coupled with design, research, production, sales, user feedback and field trials.

It is anticipated that robotics will fundamentally reduce labour costs by replacing a large proportion of routine roles. The Copenhagen study shows that UK adoption of industrial automation will produce a long-term increase in productivity of 22% and workforce increase of 7.4% as staff are re-skilled and moved to higher skilled roles. In this context, iSee's will be extremely significant since the project will have impact on the development of future service robots unlocking new industries. Vision for service robots is a significant challenge, and if we can deploy an effective sensorised solution, we have the potential to enable a new wave of startups creating vision-enabled service robots across multiple market domains.

People
iSee will facilitate the development of new vision and robotics skills in the research associates and CVAS academic staff. People with robotics and autonomous systems expertise are in high demand in both industry and academia and are significant and economic contributors. Likewise, the ability to deploy robots to perform a wider range of repetitive manual tasks will reduce the incidence of industrial injuries due to tiredness and boredom, and, hence, improve the quality of people's working environments.

Society
Robots are also key to addressing social challenges in high wage economies, e.g. increasing healthcare demands and the aging population. The enhanced reliability that iSee will deliver is essential for disruptive new robotic technologies that are not currently possible such as hospitals and care facilities. A new generation of robots may be deployed in assistive and care roles, which could have significant impact on social care and aging society challenges. End effectors that can see mean that robots can work in areas where illumination cannot be controlled and in areas where access is constrained or dangerous such as inside storage shelving or inside large workpieces, and land mine defusing or bomb disposal.