ROBOTIC TELEPRESENCE: EFFICIENT ENCODING BASED ON INTELLIGENT SCENE IDENTIFICATION
Lead Research Organisation:
University of Strathclyde
Department Name: Electronic and Electrical Engineering
Abstract
Telepresence robots are often used in remote locations where it would be impractical for the operator to travel to. Telepresence can also be used in hazardous areas where it would be unsafe for a human to enter.
For effective telepresence to be achieved the operator must have a clear view of the robots surroundings. This requires a wide field of view and high resolution video which can be difficult to transmit over a network.
This PhD will focus on reducing the bandwidth requirements of telepresence and video conferencing by identifying objects within the scene and altering the compression rate accordingly. For example in video conferencing the users main focus is likely to be on people in the scene. Faces should therefore be transmitted in high resolution and at a high frame rate. A plant in the background which has leaves blowing in the wind does not provide relevant information to the user. The plant can therefore be encoded at a higher compression rate and have its movement ignored between frames. Presentation slides and whiteboards should be viewed in high quality but a high frame rate is not required.
Telepresence robots often have to operate at high latencies which can make them difficult to manoeuvre. To counteract this a wide field of view camera can be used. The resulting image of a fisheye lens with FOV up to 280 degrees can be disorientating to view even after undistortion. A subset of this image will be encoded as the standard video stream. The increased compression rate will allow the scene around the main video to be built up over multiple frames. This will provide smoother control to telepresence operators. For example A turn to the right could be simulated in real time instead of being viewed after a potentially long transmission delay.
For effective telepresence to be achieved the operator must have a clear view of the robots surroundings. This requires a wide field of view and high resolution video which can be difficult to transmit over a network.
This PhD will focus on reducing the bandwidth requirements of telepresence and video conferencing by identifying objects within the scene and altering the compression rate accordingly. For example in video conferencing the users main focus is likely to be on people in the scene. Faces should therefore be transmitted in high resolution and at a high frame rate. A plant in the background which has leaves blowing in the wind does not provide relevant information to the user. The plant can therefore be encoded at a higher compression rate and have its movement ignored between frames. Presentation slides and whiteboards should be viewed in high quality but a high frame rate is not required.
Telepresence robots often have to operate at high latencies which can make them difficult to manoeuvre. To counteract this a wide field of view camera can be used. The resulting image of a fisheye lens with FOV up to 280 degrees can be disorientating to view even after undistortion. A subset of this image will be encoded as the standard video stream. The increased compression rate will allow the scene around the main video to be built up over multiple frames. This will provide smoother control to telepresence operators. For example A turn to the right could be simulated in real time instead of being viewed after a potentially long transmission delay.
Organisations
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509760/1 | 01/10/2016 | 30/09/2021 | |||
| 1803470 | Studentship | EP/N509760/1 | 01/10/2016 | 31/03/2020 | Liam McDonald |
| Description | That modern video codecs (H.265) do not benefit sufficiently from scene aware video encoding (separating foreground and background). That commercial photogrammetry solutions are not suitable for pipe bore inspections, the lack of loop closures leads to severe deterioration of scale. A method of virtually centralising cameras within pipework and producing undistorted panoramic stitches of the interior surface of pipes. |
| Exploitation Route | This work is finding use in the visual inspection of pipework. |
| Sectors | Chemicals,Construction,Energy |
| Description | The findings are being used in a Oil and Gas Innovation Centre (OGIC) project looking at pipe inspection. This is a joint project with Inspectahire and the University of Strathclyde. |
| First Year Of Impact | 2019 |
| Sector | Construction,Digital/Communication/Information Technologies (including Software),Energy |
| Impact Types | Economic |