Enabler for Next-Generation Mobile Video Applications

Lead Research Organisation: University of the West of Scotland
Department Name: School of Computing

Abstract

The High Efficiency Video Coding (HEVC) standard under development is the next-generation video compression standard, which is destined to revolutionise the world of video applications thanks to the expected high compression efficiency compared with the current standard. Nevertheless, to turn the vision into reality, substantial research efforts are necessitated since HEVC is still at its early standardisation stage and there is little research on networked HEVC applications. The proposed project targets mobile HEVC video delivery in an emerging and challenging networking paradigm, multihomed mobile networks, which are moving networks with a number of mobile users onboard (e.g., passengers on public transport vehicles). A multihomed mobile network has access to multiple complementary wireless networks such as 3G and beyond, WiFi and WiMAX, each of which provides a different network path to onboard multihomed mobile routers managing the multihoming and the mobility of the whole moving network. Compared with individual multihomed mobile hosts, a distinctive advantage of a multihomed mobile network is that the mobile users' devices do not need being multihomed or mobility aware. Such multihoming and mobility transparency substantially reduces the requirements on users' devices and the costs for deploying and operating such a system.
The proposed research will attempt to enable bi-directional real-time video transmission between the mobile users within a multihomed mobile network and the infrastructure to support various applications and services for business, entertainment, safety, security or other purposes. A typical application scenario is in the public transportation context. In this case, the onboard passengers receive high-quality downlink HEVC-based video streamed from an HEVC media server. Meanwhile, the live onboard scenes are captured by a camera, encoded by an HEVC encoder, and streamed upwards to a monitoring station for public security and incident management. It is expected that the mobile user's quality of experience (QoE) will be significantly improved compared with latest standards H.264 Advanced Video Coding (AVC) and Scalable Video Coding (SVC) through the proposed research.
The proposed research can be easily adopted in a range of application scenarios such as intelligent transportation systems, emergency/urgency management, telemedicine, personal area networks, and home and camping entertainment. Therefore, there is a strong technological and economic need for further research into the theory and application of the proposed topic. Moreover, joint contributions with Qualcomm to the JCT-VC meetings will enable us to set benchmarks in mobile HEVC applications and contribute to the standardisation of HEVC, which can lead to a long-lasting global impact. Dissemination and exploitation plans will promote the research outcome to the academic community as well as companies (such as AIMS (INT) Ltd., another partner of the project) related to video networking applications to exploit potential commercial product development.

Planned Impact

Video content has become a ubiquitous presence in everyday life with ever-growing demands for higher quality at any time, any location and an expanding variety of user equipment. The next-generation video coding standard HEVC is destined to change the world of mobile video applications thanks to its doubled compression efficiency compared with the latest H.264/AVC standard. The proposed research will design and implement a real-time high-quality HEVC video delivery system for mobile users within a moving network that can access to multiple wireless access networks simultaneously.
Such a mobile HEVC system can be applied in numerous applications that will directly enhance our well-being. An immediate application scenario is in the transport industry with intelligent transportation systems. The downlink video streaming to the passengers from content providers can be used for entertainment and business purposes, and such applications can create new business models for the public transportation operators and content providers. By offering such value-added services, more people will be attracted to use public transportation instead of private cars, thereby helping to protect the environment with less carbon emission. Furthermore, the onboard live video can be streamed to a surveillance centre for monitoring passenger safety etc.
The application platforms can be extended beyond public transportation to public service vehicles (e.g., NHS ambulances) and rescue teams for telemedicine, emergency/urgency management and post-disaster recovery. Bi-directional real-time high-quality videos between the emergency site and the control centre (hospitals etc.) are highly useful in assisting timely decision making and rescue operations.
The mobile network users (such as commuters, travellers and tourists) will be able to enjoy real-time high-quality next-generation video applications enabled by this research thanks to its user's quality of experience (QoE) oriented design approach. This will introduce great fun to the journey for general travellers, and greatly facilitate tourists to learn the interested places. Mobile workers that need video instructions will also enjoy improved experience.
Moreover, the mobile network paradigm for a group of users can be scaled down to a personal area network, where a single mobile user can utilise his/her multihomed smartphone as the mobile gateway router to manage his/her other personal electronic devices such as laptop, tablet PC and camera. Finally, although the more challenging mobile scenario is targeted, the proposed system will work for static premises such as a camping van or an actual business or residential house. In these cases, the multihomed mobile routers are fixed in the premise to provide multihoming for improved QoE whilst the mobility support functionality is simply not triggered.
Overall, many companies related to video networking applications can exploit potential commercial products based on the outcome of the proposed research. For instance, the applicant has recently helped AIMS (INT) Ltd., a hi-tech company in the region, in developing an IPTV testing platform based on the H.264/AVC standard. It will be a great selling point if the platform is upgraded to support the next-generation standard HEVC.
In addition to the above socio-economic benefits and the academic benefits mentioned, the proposed research will be a valuable asset to the JCT-VC standardisation team, who will be interested to learn the project's results such as the performance of the candidate HEVC codec of different configurations for mobile network users. Such timely knowledge will help the JCT-VC team to determine the choice of future standard HEVC codec for mobile user. Moreover, if any of our contributions are included in the final specification of the HEVC standard, long-lasting global impact can be ensured as video chipmakers and network operators will use the standard in a wide gamut of their products.

Publications

10 25 50
publication icon
I. Irondi (2019) Efficient QoE-Aware Scheme for Video Quality Switching Operations in Dynamic Adaptive Streaming in ACM Transactions on Multimedia Computing, Communications, and Applications

publication icon
Nightingale J (2014) The impact of network impairment on quality of experience (QoE) in H.265/HEVC video streaming in IEEE Transactions on Consumer Electronics

publication icon
Nightingale J (2018) 5G-QoE: QoE Modelling for Ultra-HD Video Streaming in 5G Networks in IEEE Transactions on Broadcasting

publication icon
Nightingale J (2013) Performance Evaluation of Concurrent Multipath Video Streaming in Multihomed Mobile Networks in International Journal of Digital Multimedia Broadcasting

 
Description Firstly, new knowledge has been generated in the field of a rang of novel cutting-edge video networking technologies based on the latest video coding standard (H.265). Detailed research outcomes have been published in over 10 peer-reviewed publications so far.
Secondly, new research opportunities have been opened up especially in the direction of real-time H.265 video applications. In addition, further research has been identified to investigate the application of the research outcomes in a wide range of use cases. The project has led to a couple of funded follow-up projects.
Thirdly, close international collaboration with the involved industry partner (Qualcomm, USA) and the academic partner (Stuttgart Media University, Germany) have been well developed through this project.
Exploitation Route The main theme of this research is video networking and processing technologies based on the latest video coding standard (H.265). The research outcomes have a wide spectrum of usage in various businesses where video applications based on the latest standard can help. Examples include (but not limited to) video surveillance for public safety and health in public transportation and smart city environments; visual information transmission and collaboration services in wireless/mobile networks for command and control systems, disaster relief and emergency services; video streaming for broadcasting, IPTV, gaming, teleconferencing, tele-presence, e-learning, e-healthcare, e-tourism and so on. The PI and his team are actively exploring the exploitation opportunities. For instance, one KTP project has been set up to investigate the accelerated video processing of H.265 for the games industry. Moreover, a project funded by UK Defence Science and Technology Laboratory (DSTL) has further explored the potential of video streaming in bandwidth-constrained tactical networks.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Communities and Social Services/Policy,Construction,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Electronics,Energy,Environment,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Culture, Heritage, Museums and Collections,Retail,Security and Diplomacy,Transport,Other

URL http://www.uws.ac.uk/research/research-institutes/computing/avcn/projects/hevc/
 
Description The findings of this project have been further explored in follow-up funded projects on video/imagery related R&D activities, including the video optimisation in the EU Horizon2020 5G-PPP project (H2020-ICT-2014-2/671672), "SELFNET: Framework for Self-Organised Network Management in Virtualized and Software Defined Networks" and so on.
First Year Of Impact 2015
Sector Digital/Communication/Information Technologies (including Software),Security and Diplomacy
Impact Types Societal,Economic,Policy & public services

 
Description NATO IST-118 Membership
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
URL https://www.cso.nato.int/ACTIVITY_META.asp?ACT=2293
 
Description CENSIS Collaborative Project
Amount £87,000 (GBP)
Organisation Innovation Centre for Sensor and Imaging Systems CENSIS 
Sector Charity/Non Profit
Country United Kingdom
Start 12/2015 
End 11/2016
 
Description MoD-DSTL
Amount £22,312 (GBP)
Organisation Defence Science & Technology Laboratory (DSTL) 
Sector Public
Country United Kingdom
Start 03/2015 
End 05/2015
 
Description SELFNET: Framework for Self-Organised Network Management in Virtualized and Software Defined Networks
Amount € 6,866,496 (EUR)
Funding ID H2020-ICT-2014-2/671672 
Organisation European Commission 
Department Horizon 2020
Sector Public
Country European Union (EU)
Start 07/2015 
End 06/2018
 
Description SliceNet: End-to-End Cognitive Network Slicing and Slice Management Framework in Virtualised Multi-Domain, Multi-Tenant 5G Networks
Amount € 7,979,030 (EUR)
Funding ID H2020-ICT-2016-2/761913 
Organisation EU-T0 
Sector Public
Country European Union (EU)
Start 06/2017 
End 05/2020
 
Description Smart Unmanned Aerial System for Real-Time Object Detection
Amount £100,000 (GBP)
Funding ID CAF440 
Organisation Innovation Centre for Sensor and Imaging Systems CENSIS 
Sector Charity/Non Profit
Country United Kingdom
Start 12/2018 
End 12/2019
 
Description UWS-BUU
Amount £80,000 (GBP)
Organisation University of the West of Scotland 
Sector Academic/University
Country United Kingdom
Start 11/2013 
End 10/2014
 
Description Cranfield 
Organisation Cranfield University
Country United Kingdom 
Sector Academic/University 
PI Contribution Based on some of the findings of this project, we have further developed a scheme to efficiently stream video over low-bandwidth tactical networks.
Collaborator Contribution Cranfield Defence and Security has collaborated with us to explore part of the findings in this project, and apply them to the tactical domain.
Impact J. Nightingale, Q. Wang, J. M. Alcaraz-Calero, I. Owens, and C. Grecos, "Enhancing visual communications capabilities in tactical networks", Proc. International Conference on Military Communications and Information Systems (ICMCIS, formerly MCC) 2015, Cracow, Poland, May 2015. (doi: 10.1109/ICMCIS.2015.7158692)
Start Year 2014
 
Description HDM-Stuttgart 
Organisation Stuttgart Media University
Country Germany 
Sector Academic/University 
PI Contribution The PI and the partner have been co-supervising PhD students, and the PI serves as the Director of Studies.
Collaborator Contribution The PI and the partner have been co-supervising PhD students, and the PI serves as the Director of Studies.
Impact The PI and the partner have been actively producing publications on HEVC video coding technologies, which is the main theme of the funded project.
Start Year 2012
 
Description NATS 
Organisation National Air Traffic Services Limited
Country United Kingdom 
Sector Private 
PI Contribution Among the prioritised topics, NATS are interested to enhance/upgrade their video services such as surveillance. My research team are actively investigating this.
Collaborator Contribution NATS are collaborating with my research team in specifying the technical requirements for enhancing/upgrading their video services.
Impact A formal strategic partnership between UWS (the host institution) and NATS was signed on 30th May 2016.
Start Year 2016
 
Description Qualcomm 
Organisation Qualcomm
Country United States 
Sector Private 
PI Contribution The PI and his team in UWS have contributed to this collaboration/partnership by addressing and advancing technologies in two of their major business areas (the next-generation video coding standard HEVC and mobile communications). The PI and his team in UWS have been leading the co-authored publications in renowned international journals and conferences regarding the developed cutting-edge technologies, which have contributed to reinforcing the leadership in these areas of the partner.
Collaborator Contribution Qualcomm is a global industry leader in mobile communications and a major contributor to the next-generation video coding standard HEVC. The contact is Dr Sergio Goma, Principal Engineer and Head of Qualcomm Multimedia R & D.Dr Goma is an internationally leading expert in video signal processing standards and techniques, and his expertise wholly complements that of the PI in the proposed research. The development of this project has benefited from being industry and standardization informed thanks to his contributions.
Impact Five joint publications have been published, which forms a part of this project's outcomes. Please refer to the publications for more details.
Start Year 2012
 
Description Thales UK 
Organisation Thales Group
Department Thales UK Limited
Country United Kingdom 
Sector Private 
PI Contribution UWS conducted R&D activities towards resolving the challenge posed by Thales UK on "Low-pixel Automatic Target Detection and Recognition", co-funded by CENSIS.
Collaborator Contribution Thales organised and executed trials to gather representative data, and offered technical advice, critical assess and assist in R&D activities.
Impact The project developed and demonstrated algorithms that are capable to automatically detect a human or vehicle and identify the vehicle type using low-resolution thermal images.
Start Year 2015
 
Description KTN/EPSRC WINES Workshop 2013 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact The poster attracted a number of participants from industry and academic to learn more about this EPSRC project and related research.

Requests for more technical details of this EPSRC project were received from universities in England and Scotland. A company also approached the team for a potential collaboration.
Year(s) Of Engagement Activity 2013
URL http://26septwines.eventbrite.co.uk/
 
Description NATO IST-118 2013-2014 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This NATO RTO Task Group IST-118 is a formal international working group on service-oriented architectures (SOA) and computing in tactical networks. The group consists of more than 10 representatives from multiple NATO countries including UK, Netherlands, Norway, Germany, Poland, and USA. Regular face-to-face meetings or teleconferences have taken place, where the PI has presented his contributions, partially based on selected outcomes of this EPSRC project.

The goal of the RTG is to investigate services and applications in tactical networks through research and experimenting with possible SOA improvements. Based on the results, the goal is to provide guidance (best practices) to make SOA applicable on battlefield disadvantaged networks in the form of a Tactical SOA Profile as Recommendations to influence the decision making in NATO operations. Research results on video applications in this context are planned to form part of the final Recommendations, partially based on research outcomes from and tools developed in this EPSRC project.
Year(s) Of Engagement Activity 2013,2014
URL https://www.cso.nato.int/ACTIVITY_META.asp?ACT=2293
 
Description SCONE Meeting 2013 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact "SCONE is the SCOttish Networking Event - an informal gathering of networking and systems researchers in and around Scotland. The goal of these meetings is to foster interaction between researchers from our various institutions." (from SCONE website) About 50 researchers including staff and research students from a number of Scottish universities (St Andrews, Edinburgh, Strathclyde, Stirling etc.) attended this half-day event, organized and hosted by the PI and the research fellow of this EPSRC project at their host institution (UWS). The PI and the research fellow jointly made an oral presentation to promote this EPSRC project, and has an interactive discussion with the audience.

This event has increased the visibility of the research of the PI's School as well as this EPSRC project. The work reported by the PI and the research fellow was well received by the audience. It has been reported that the specific topics focused on in this EPSRC project have since then become proposed topics for PhD programs in other Scottish universities.
Year(s) Of Engagement Activity 2013
URL http://scone.cs.st-andrews.ac.uk/wiki/Meeting29102013
 
Description SICSA Demofest 2013 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact It is estimated that over 50 people from academia and industry attended this event, where the PI and the research follow working with him presented two posters promoting this EPSRC project. They both were also interviewed by the organizer and their interviews were recorded by professional cameramen invited by the organizer for this event.

The presentation from the PI and the research follow attracted great interest from Codeplay, an Edinburgh-based high-tech company working on closely related areas (video processing of latest standards). Follow-up meetings were then held between the PI and his team, leading to a KTP project to exploit some of the research outcomes of this EPSRC project. This KTP project has been recently successfully granted, and a KTP associate is being recruited to undertake this two-year project.
Year(s) Of Engagement Activity 2013
URL http://www.sicsa.ac.uk/knowledge-exchange/industry-collaboration/demofest/
 
Description VQEG Meetings 2014 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact In the two VQEG meetings held this year, the PI presented the EPSRC project and outcomes of the research as part of the working group's report under AVHD (Audio-Video High Definition). Interactive discussions took place between the PI and the audience as partially noted in the meetings' minutes.

VGEG (Video Quality Experts Group) in an international expert group on video quality evaluation, modeling and related areas. "VQEG submits reports to and liaises with regional and international standardization bodies in particular the ITU (International Telecommunication Union)." (from VQEG website) The PI was encourage to make further contributions to the VQEG's research schedule in adaptive streaming by further exploiting the research findings through the EPSRC project and follow-up research, and potentially contribute to the standardization in this area.
Year(s) Of Engagement Activity 2014
URL http://www.its.bldrdoc.gov/vqeg/vqeg-home.aspx