Optical fibre networks underpin the digital communications infrastructure
Lead Research Organisation:
University College London
Department Name: Electronic and Electrical Engineering
Abstract
The next research challenge is to introduce intelligence so that they able to dynamically provide capacity where and when it is needed - transforming next-generation information infrastructure. Cloud network traffic is expected to more than double every two years and new services require very low latency requirements. Research will focus on developing new approaches to the design of network topologies and architectures, tailored to application requirements: throughput, scalability, latency and resilience. Optimal topologies will need to be adaptive and be tailored to applications requirements on capacity and delay, physical properties of the transmission channels and the information from intelligent transceivers on network state. Research will explore heuristic algorithms for wavelength routing and route selection and explore if machine learning approaches can achieve more optimal topologies. The work will also have experimental element in the development of the TRANSNET Virtual Lab and experimental network scenarios which can be used to demonstrate soe of the theoretical concepts. This work is partially supported by Microsoft Research through the Research Alliance in 'Optics for the Cloud' and this PhD project is within UCL's Optical Networks Group to work in this area, focusing on different challenges relating to optical networking in the Cloud. The research is also associated with EPSRC TRANSNET (Transforming optical networks - building an intelligent optical infrastructure) programme grant: http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/R035342/1. This research is firmly within the EPSRC ICT Theme covering Artificial Intelligence, Infrastructure, Optical Communications and Digital Signal Processing.
People |
ORCID iD |
| Polina Bayvel (Primary Supervisor) | |
| Robin Matzner (Student) |
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/R513143/1 | 01/10/2018 | 30/09/2023 | |||
| 2230989 | Studentship | EP/R513143/1 | 01/10/2019 | 30/03/2024 | Robin Matzner |
| Description | There have been some key findings regarding understanding the relationship between throughput, topology structure and physical properties, as well as then the design of optical networks: - We have seen that by investigating an analytical metric that incorporates the structural and physical properties of optical networks, one can linearly correlate to maximum achievable throughput of networks. This has pushed our understanding of how topology structure and physical properties of optical networks impact the throughput they can sustain. - Further research looked at learning the relationship between the topology and the throughput from previous data via geometric deep learning and applying message passing neural networks (MPNN). We generated 3 datasets spanning 240,000 topologies and their associated throughputs. Here we saw that geometric deep learning was able to learn the relationship accurately between the throughput and the topology with high linear correlation and reducing the computational time required by 5 orders of magnitude on average. For this to be widely applicable however more data and more variance in this data is required to account for different traffic demands and topology structures. - Using the analytical metric (DWC) we designed a parralelised meta-heuristic, which can design large scale (100 node) optical core networks efficiently and optimally. Here the key finding was that by parralelising the design process and running multiple meta-heuristics at the same time, we were able to both design better networks and also reduce the computational time needed to design these networks. |
| Exploitation Route | Others can take the analytical and learnt models from this research and directly apply them to their own optimisation algorithms that they are working on for designing optical networks. They might take the tools and apply them to other networking problems. Industry could take the approach of optical network design that we developed to design new optical networks that might be more efficient and better connected, as they might not understand the impact of network structure on the performance metrics of optical networks. |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Title | Maximum Achievable Throughput Predictor |
| Description | The maximum achievable throughput predictor is a message passing neural network (MPNN) trained on over 240 000 graphs for optical networks between 10 and 100 nodes. It is able to take in an optical network and predict the maximum achievable throughput that network will be able to achieve without evaluating the routing and wavelength assignment, within ms. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2021 |
| Provided To Others? | No |
| Impact | This tool majorly reduces the time needed to evaluate the maximum amount of throughput an optical core network can route (5 orders of magnitude less). This will allow one to use throughput within the optimisation of optical core networks in the future, which was not possible before. Therefore expanding their throughput and improving our digital communications infrastructure. |
| Title | New optical network design methodology to design large scale optical core networks |
| Description | In this method we researched a distributed meta-heuristic approach to design large (100 node) optical core networks. This method was validated against random and genetic algorithm baselines and seen to considerably increase throughput in networks. This can be used to generate good networks that can be benchmarked or used within other optical core network simulation studies. In addition, a computationally efficient analytical method for calculating the performance of optical core networks (normally an NP-hard problem) was developed and could be used within any optimisation method that researchers might be investigating. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2023 |
| Provided To Others? | Yes |
| Impact | Other researchers can use the demand weighted cost (DWC) as a computationally efficient performance metric or cost function to optimise optical core networks further. In addition, the design method that we published can be used to produce networks that are good topologies and could be used to standardise network studies within optical core networking research. |
| URL | https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10032971 |
| Title | Optical Network Geometric Generative Graph Model |
| Description | A geometric generative graph model, that models random geometric graphs so that they have realistic structure and physical properties compared to real optical core networks. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2021 |
| Provided To Others? | Yes |
| Impact | A lot of networking research in optical core networking is tested on a few single real topologies, however to bring statistical significance to results we need to test on many different networks. Often models like Barabasi-Albert graphs or Waxman graphs are used, however these do not simulate the optical network structure realistically. The model generated (SNR-BA) allows for researchers to generate many realistic graphs for optical core network modelling. Giving us more siginificant results and deeper understanding of the networks that carry our day to day digital information. |
| Title | Alpha Model |
| Description | Trained MPNN model to infer throughput values from a optical network topology with nodes beteween 10 - 15. This model has been trained with the Alpha training dataset also available in this data repository, where the throughput labels are calculated using ILP solutions to the maximal routing and wavelength assignment problem. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/model/Alpha_Model/21689105/1 |
| Title | Alpha Test Datset |
| Description | This set of data houses the 6000 graphs used for testing the beta model that has been trained on the beta dataset. It houses topologies generated via SNR-BA [1] with nodes scattered uniformly randomly with a minimum radius of 100km between them over a grid the size of north america. The throughput labels are calculated via maximising the routing and wavelength assignment with zero blocking using an integer linear programming formulation and implementing the physical layer impairments using the gaussian noise model. [1] R. Matzner, D. Semrau, R. Luo, G. Zervas, and P. Bayvel, 'Making intelligent topology design choices: understanding structural and physical property performance implications in optical networks [Invited]', J. Opt. Commun. Netw., JOCN, vol. 13, no. 8, pp. D53-D67, Aug. 2021, doi: 10.1364/JOCN.423490. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Alpha_Test_Datset/21689078/1 |
| Title | Alpha Training Dataset |
| Description | Training dataset for nodes between 10-15 nodes with throughput labels. The graphs are generated by the SNR-BA [1] model with nodes scattered uniformly over a grid the size of north america with mimum distances between nodes set to 100km. The throughput labels are generated by maximising the routing and wavelength assignment by a integer linear programming formulation at zero blocking and calculating the physical layer impairements via the gaussian noise model. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Alpha_Training_Dataset/21689072/1 |
| Title | BA Test Datset |
| Description | This dataset consists of 25-45 node graphs generated via BA [1] with uniformly randomly scattered nodes over a grid the size of north america with a minimim of 100km between nodes. This is a test dataset to test how the model reacts to different structures of graphs. [1] A.-L. Barabasi and R. Albert, 'Emergence of scaling in random networks', Science, vol. 286, no. 5439, pp. 509-512, Oct. 1999, doi: 10.1126/science.286.5439.509. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/BA_Test_Datset/21689075/1 |
| Title | Beta Model |
| Description | Trained MPNN model to infer throughput values from a optical network topology with nodes beteween 25 - 45. This model has been trained with the Beta training dataset also available in this data repository. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/model/Beta_Model/21689111/1 |
| Title | Beta Skewed Traffic Test Dataset - gamma=0.2 |
| Description | This set of data houses a test set of 1000 graphs with locally skewed traffic at a rate of gamma=0.2. The throughput labels are calculated with the same methodology as the other beta sets just subjected to different traffic conditions. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Skewed_Traffic_Test_Dataset_-_gamma_0_2/21689093/1 |
| Title | Beta Skewed Traffic Test Dataset - gamma=0.4 |
| Description | This set of data houses a test set of 1000 graphs with locally skewed traffic at a rate of gamma=0.4. The throughput labels are calculated with the same methodology as the other beta sets just subjected to different traffic conditions. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Skewed_Traffic_Test_Dataset_-_gamma_0_4/21689096/1 |
| Title | Beta Skewed Traffic Test Dataset - gamma=0.6 |
| Description | This set of data houses a test set of 1000 graphs with locally skewed traffic at a rate of gamma=0.6. The throughput labels are calculated with the same methodology as the other beta sets just subjected to different traffic conditions. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Major impact across the TRANSNET programme within the TRANSNET Virtual Lab - enabling a wide range of network topologies for network design, training, analysis and quantifiable comparisons. This has enriched the area of network design and is expected to be widely used by partners and well beyond the programme. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Skewed_Traffic_Test_Dataset_-_gamma_0_6/21689084/1 |
| Title | Beta Skewed Traffic Test Dataset - gamma=0.8 |
| Description | This set of data houses a test set of 1000 graphs with locally skewed traffic at a rate of gamma=0.8. The throughput labels are calculated with the same methodology as the other beta sets just subjected to different traffic conditions. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Skewed_Traffic_Test_Dataset_-_gamma_0_8/21689081/1 |
| Title | Beta Skewed Traffic Test Dataset - gamma=1.0 |
| Description | This set of data houses a test set of 1000 graphs with locally skewed traffic at a rate of gamma=1.0. The throughput labels are calculated with the same methodology as the other beta sets just subjected to different traffic conditions. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Skewed_Traffic_Test_Dataset_-_gamma_1_0/21689099/1 |
| Title | Beta Test Dataset |
| Description | This set of data houses the 5000 graphs used for testing the beta model that has been trained on the beta dataset. It houses topologies generated via SNR-BA [1] with nodes scattered uniformly randomly with a minimum radius of 100km between them over a grid the size of north america. The throughput labels are calculated via maximising the routing and wavelength assignment with zero blocking using first-fit k-shortest-paths and implementing the physical layer impairments using the gaussian noise model. [1] R. Matzner, D. Semrau, R. Luo, G. Zervas, and P. Bayvel, 'Making intelligent topology design choices: understanding structural and physical property performance implications in optical networks [Invited]', J. Opt. Commun. Netw., JOCN, vol. 13, no. 8, pp. D53-D67, Aug. 2021, doi: 10.1364/JOCN.423490. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Test_Dataset/21689090/1 |
| Title | Beta Training Dataset |
| Description | This training dataset included optical network topologies that are generated via SNR-BA method [1] with nodes scattered uniformly randomly over a grid the size of the north american continent. Here there is a minimum radius that is adhered to (100km) between the nodes. The nodes are between scales of 25-45 nodes. The routings of the network are computed under uniform bandwidth conditions with the first-fit k-shortest-path (FF-kSP) algorithm and sequential loading (SL) until the maximum state of the network is found at zero blocking. The Gaussian noise (GN) model is used to calculate the signal-to-noise ratio of paths and the total throughput of the network. This throughput is given as a training label. [1] R. Matzner, D. Semrau, R. Luo, G. Zervas, and P. Bayvel, 'Making intelligent topology design choices: understanding structural and physical property performance implications in optical networks [Invited]', J. Opt. Commun. Netw., JOCN, vol. 13, no. 8, pp. D53-D67, Aug. 2021, doi: 10.1364/JOCN.423490. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Beta_Training_Dataset/21695687/1 |
| Title | ER Test Dataset |
| Description | This dataset consists of 25-45 node graphs generated via ER [1] with uniformly randomly scattered nodes over a grid the size of north america with a minimim of 100km between nodes. This is a test dataset to test how the model reacts to different structures of graphs. [1] P. Erdos and A. Renyi, 'On the Evolution of Random Graphs', in Publication of the Mathematical Institute of the Hungarian Academy of Sciences, 1960, pp. 17-61. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/ER_Test_Dataset/21689087/1 |
| Title | Gamma Model |
| Description | Trained MPNN model to infer throughput values from a optical network topology with nodes beteween 55 - 100. This model has been trained with the Gamma training dataset also available in this data repository. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/model/Gamma_Model/21689108/1 |
| Title | Gamma Test Dataset |
| Description | This set of data houses the 5000 graphs used for testing the gamma model that has been trained on the gamma dataset. It houses topologies generated via SNR-BA [1] with nodes scattered uniformly randomly with a minimum radius of 100km between them over a grid the size of north america. The throughput labels are calculated via maximising the routing and wavelength assignment with zero blocking using first-fit k-shortest-paths and implementing the physical layer impairments using the gaussian noise model. [1] R. Matzner, D. Semrau, R. Luo, G. Zervas, and P. Bayvel, 'Making intelligent topology design choices: understanding structural and physical property performance implications in optical networks [Invited]', J. Opt. Commun. Netw., JOCN, vol. 13, no. 8, pp. D53-D67, Aug. 2021, doi: 10.1364/JOCN.423490. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Gamma_Test_Dataset/21695996/1 |
| Title | Gamma Training Dataset |
| Description | This training dataset included optical network topologies that are generated via SNR-BA method [1] with nodes scattered uniformly randomly over a grid the size of the north american continent. Here there is a minimum radius that is adhered to (100km) between the nodes. The nodes are between scales of 55-100 nodes. The routings of the network are computed under uniform bandwidth conditions with the first-fit k-shortest-path (FF-kSP) algorithm and sequential loading (SL) until the maximum state of the network is found at zero blocking. The Gaussian noise (GN) model is used to calculate the signal-to-noise ratio of paths and the total throughput of the network. This throughput is given as a training label. [1] R. Matzner, D. Semrau, R. Luo, G. Zervas, and P. Bayvel, 'Making intelligent topology design choices: understanding structural and physical property performance implications in optical networks [Invited]', J. Opt. Commun. Netw., JOCN, vol. 13, no. 8, pp. D53-D67, Aug. 2021, doi: 10.1364/JOCN.423490. |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Not aware of any impact. |
| URL | https://rdr.ucl.ac.uk/articles/dataset/Gamma_Training_Dataset/21696008/1 |
| Description | Aston-UCL application of message passing optimisation to optical network routing and wavelength assignment |
| Organisation | Aston University |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Benchmarking the application of the message passing algorithm against baseline heuristics and running the simulations. |
| Collaborator Contribution | The developing of the message passing algorithm for the application to routing and wavelength assignment problems. |
| Impact | OFC 2022 Conference paper |
| Start Year | 2020 |
| Description | Microsoft collaboration |
| Organisation | Microsoft Research |
| Department | Microsoft Research Cambridge |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | John Nuqui, PhD student, recruited at our Cambridge site. He has made significant contributions to our research agenda. |
| Collaborator Contribution | Microsoft have provided the TRANSNET team with excellent expertise, in knowledge exchange particularly. |
| Impact | Ongoing |
| Start Year | 2019 |
| Description | Google Networking Summit - Lightning Talk |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Invited lightning talk at the Google Networking Summit. Presentation of recent research on intelligent optical network design using graph neural networks, which raised awareness to a broader audience of the problems that are faced in the physical layer optimisation of the optical core networks. Sparked questions by various researchers in the field. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Poster at TOP conference London |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Poster exhibited at the TOP conference in London. Engagement with industry experts and academic researchers alike. Made people aware of the applications of graph neural networks within optical core network design. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Poster given at European Conference on Optical Communications (ECOC) in Basel |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | A poster titled "Expanding Graph Neural Networks for Ultra-Fast Optical Core Network Throughput Prediction to Large Node Scales" was presented at the European Conference on Optical Communications in Basel. Here I interacted with other colleagues from around the world. Here many discussions were had with other students and researchers in my field and on the techniques I had used for this specific problem. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://ieeexplore.ieee.org/abstract/document/9979635 |
| Description | Presentation at Optical Network Design and Modelling Conference in Warsaw |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | A talk on "Ultra-fast Optical Network Throughput Prediction using Graph Neural Networks", where work on reducing computational complexity of calculating NP-hard performance metrics (i.e. maximum achievable throughput) was disseminated to other participants from around Europe. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9782853 |