Transfer Learning for Robust, Resilient and Transferable Cyber Manufacturing Systems

Lead Research Organisation: Swansea University
Department Name: School of Engineering

Abstract

Digital Manufacturing relies on pervasive and ubiquitous use of Information and Communication Technology (ICT), sensors, intelligent robots to deliver the next generation of intelligent, co-operating and interconnected manufacturing systems. The research is aimed to improve techniques that can be used to develop digitalised manufacturing systems to reduce existing inefficiencies in production processes that impact on production costs, unplanned downtime, quality and yields. This is not only detrimental to manufacturing businesses but has a negative impact on the UK Economy. The current productivity levels of UK manufacturers and suppliers is lagging behind global competitors and prevents the UK from successfully competing with other countries in the manufacturing domain - which is vital to keep businesses and jobs in the UK rather than relocate production abroad.

The UK Government wants to increase the strength of the UK Manufacturing Sector. A key means of doing this is the widespread adoption of industrial digital technologies (IDT). Cyber Manufacturing Systems (CMS) are the building blocks of digitalised manufacturing and generate vast amount of data that can be used for real time decision making to achieve optimised performance through predictive and prescriptive analytics. The latter are techniques that use, combine and analyse available data to develop computational models that can predict future outcomes and determine the best course of action.The research, under the fellowship, solves some of the existing problems in this area (CMS), developing new techniques and resources for predictive and prescriptive analytics with the potential to increase efficiency, accuracy and productivity of manufacturing processes. Businesses are therefore more likely to adopt IDTs and improve profitability and sustainability and provide high-quality jobs in a thriving part of the economy.

This project will study novel and robust data analytics methods that will enable to build predictive models that take into account uncertainty, complexity and dynamic behaviour of productions systems. The project will involve:

Objective 1 - develop algorithms that can reuse previously acquired data/knowledge to build more accurate predictive models that work well in the presence of noise (i.e. 'robust'), are able to adapt to changes over time (i.e. 'resilient') and can be scaled up across multiple factories (i.e. 'transferable').

Objective 2 - develop and test novel non-parametric methods for estimation of uncertainty and risks associated to a decision to enable real time mission and safety critical decision making (both automated and human driven) based on predictions.

Objective 3 - iteratively develop, deploy and test predictive and prescriptive models in real and simulated industrial scenarios to obtain acceptable level of performance, usability and robustness.

There will be significant involvement from industrial collaborators who will provide labelled and aggregated datasets for testing the proposed methods through computer simulations and enable feasibility studies to be conducted in factory environments.

The outcomes of the research, as mentioned above, are ultimately to improve the quality of products, achieving less wastage and unnecessary costs. Through increased adoption of IDTs, the production of goods will, importantly, be more efficient, reliable and profitable. This will support the regeneration of the Manufacturing Sector and boost the global competitiveness of the UK.

Planned Impact

The UK Government has crystallised the necessity for growth in the manufacturing sector via the adoption and integration of industrial digital technologies [1, 2]. The Government's aspiration is for the UK to become a World leader in Digital Manufacturing by 2030. However, at present the manufacturing sector has a 20% deficit in productivity compared to its main European competitors. Against this backdrop, my research will help to combat the UK productivity gap, stimulating the incorporation of digital technology within UK industry. Cyber Manufacturing Systems (CMS) are the foundations of digitalised production systems - my research will develop new machine learning techniques and toolkits for robust, resilient and transferable CMS. This will manifest in improved understanding of the benefits of IDTs adoption, and their value,and inspire confidence for manufacturers and suppliers to invest in this area. The beneficiaries of my research are a) the UK Economy, b) Organisations within the UK and International Manufacturing Sector, c) members of the public.

The UK Economy will ultimately benefit from my research, as aforementioned, via uptake in industry and the subsequent increase in productivity within the Manufacturing Sector (also potentially leading to job creation and re-shoring to the UK). This will foster global competitiveness, whilst helping to realise well publicised political aims.

Organisations within the UK and International Manufacturing Sector, beginning with my industry collaborators (TATA Steel, Crown Packaging and Vortex IoT), will prosper in terms of economic enhancement. My work during this Fellowship will bequeath new and improved processes that can be implemented in multiple manufacturing domains. This will achieve increased accuracy, efficiency and flexibility, less wastage of products through quality improvement, reduced costs and enhanced profit margins through failure reduction. Corporate performance will be markedly improved. This will have strong impact, particularly on SMEs, supporting sustainability, investment and expansion.

Members of the public will be engaged, through the outreach activity that is integrated in the Fellowship. This includes Science Festivals and initiatives such as the Fujitsu Young Ambassadors Programme. The dissemination of the research will acutely increase public awareness and comprehension of Engineering and the relevance of industrial digital techniques in the Manufacturing Sector. Specific audiences that will be targeted, in alignment with my commitment to developing the future workforce and supporting economic regeneration, will be School Children and those in Further Education.

In short, the research will produce a range of far reaching impacts that will culminate to drive forward economic prosperity; help to promote and propel the incorporation of digital technologies within the Manufacturing arena; and complement the Governments agenda for achieving productivity growth and economic competitiveness.

[1] UK Government. Building Our Industrial Strategy Green Paper. 2017 (https://beisgovuk.citizenspace.com/strategy/industrial strategy/supporting_documents/buildingourindustrialstrategygreenpaper.pdf)
[2] Group IDRW, Made Smarter Review 2017 (https://www.gov.uk/government/publications/made-smarter-review)

Publications

10 25 50
 
Description So far we have identified several use cases for the application of machine learning to improve decision making capabilities in manufacturing. These use cases focus on improving the performance of manufacturing processes in different domains (steel and beverage can manufacturing) through the use of advanced predictive analytics. A novel deep learning architecture has been proposed for multi-step ahead prediction of machine speed using real world data from an UK manufacturing plant.
Exploitation Route The Fellowship findings can help other organizations to tackle their productivity gap through adoption of Industrial Digital Technologies. By working with companies in different domains, the research aims to develop novel data driven approaches and transfer knowledge across domains, helping UK manufacturers to increase the uptake of data analytics and ML technologies. This will help manufacturers to develop smart factories that can adapt to changes more rapidly and can self-optimise, ultimately leading to increasing efficiency and improved profitability.
Sectors Manufacturing, including Industrial Biotechology

 
Description The grant research activities, currently in progress, have focused on developing novel Machine Leaning algorithms to support decision making in manufacturing to improve the quality of products, achieving less wastage and avoid unnecessary costs. The work done so far and knowledge exchange have helped partners organizations to improve their understanding of the benefits of adoption of big data technologies in their factories. As a result, partners organisations are becoming more confident to invest in this area. Through active engagement with partners organizations, we have identified several use cases to evaluate the effectiveness and robustness of Machine Leaning algorithms and related methodologies to improve the efficiency of manufacturing processes. We have actively engaged with wider industries and professional network through participation and organisation of knowledge exchange events, leading to increased awareness of the potential of Industrial Digital Technologies to drive improvements in the Manufacturing Sector.
Sector Manufacturing, including Industrial Biotechology
Impact Types Societal,Economic

 
Description EPSRC Centre for Doctoral Training in Enhancing Human Interactions and Collaborations with Data and Intelligence Driven Systems
Amount £4,986,846 (GBP)
Funding ID EP/S021892/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2019 
End 09/2027
 
Description Innovate UK
Amount £255,653 (GBP)
Funding ID TS/S015183/1 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 04/2019 
End 04/2021
 
Title 2D-Convolutional LSTM Autoencoder 
Description The research team has developed and published a novel deep ConvLSTM autoencoder architecture for machine speed prediction in a smart manufacturing process. The model has been published in IEEE Transactions on Industrial Informatics (IF 7.377) "A. E. Essien and C. Giannetti, "A Deep Learning model for Smart Manufacturing using Convolutional LSTM Neural Network Autoencoders," in IEEE Transactions on Industrial Informatics". 
Type Of Material Computer model/algorithm 
Year Produced 2020 
Provided To Others? Yes  
Impact This is the first end to end deep learning model that has been developed for multi-step ahead machine speed time series prediction. In smart factories, machine speed prediction can be used to dynamically adjust production processes, enabling improvement of production scheduling and planning. The model has been validated in a case study using real-world data obtained from a metal packaging plant in the UK. 
 
Description Collaboration with Crown Holdings 
Organisation Crown Packaging UK
Country United Kingdom 
Sector Private 
PI Contribution Research is currently being undertaken to develop novel Machine Learning techniques to support root cause analysis in can production processes to reduce waste and avoid unnecessary costs. Contributions include: development of two use cases, regular meetings, knowledge exchange and visits to the can making plant and R&D offices. The main outcome of the plant visit was to develop awareness of the possible application of big data technologies for improvement of can making processes and the development of two use use cases. The work up to date has resulted in publications of two conference papers and one journal paper.
Collaborator Contribution Crown Technology have actively contributed by providing data sets and participating to brainstorming sessions. Crown also provided access to their facilities.
Impact Knowledge Exchange - The knowledge exchange has been useful to Crown R&D and plant team to better understand the potential in developing advanced predictive models that can support decision making in their plants.
Start Year 2018
 
Description Collaboration with Vortex IoT for LiDAR object detection 
Organisation Vortex IoT
Country United Kingdom 
Sector Private 
PI Contribution Dr Giannetti is actively collaborating with Vortex IoT in the development of Machine Learning applications for LiDAR object detection. The research team includes a postgraduate student sponsored by Vortex IoT through the M2A academy.
Collaborator Contribution Engagement in meetings and industrial supervision of postgraduate research student.
Impact Knowledge exchange and initial research in LiDAR object detection.
Start Year 2019
 
Description Tata Steel Collaboration 
Organisation TATA Steel
Country India 
Sector Private 
PI Contribution Research is currently being undertaken to develop novel Machine Learning techniques to support decision making in the steel manufacturing industry. Contributions so far include the co-development of use cases in multiple production areas, gathering of user requirements and knowledge exchange activities through attendance to workshops. The research team has provided advice and support to Tata Steel employees, contributing to a use case to improve stability of Blast Furnace processes through the use of Advanced Data Analytics.
Collaborator Contribution TATA has actively engaged in meetings and provided real world data sets and access to their facilities.
Impact The insights generated through the proposed data analytics technique have allowed the company to modify the process, achieving improvement of quality and reduction of costs.
Start Year 2018
 
Description CHERISH-DE funded outreach activity at Cambridge University 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Undergraduate students
Results and Impact Participation to the development of an Outreach activity with colleagues from other institutions (Cambridge University, De Montford University, Queen's University Belfast), undertaking chemistry experiments using bubble wrap and mobile phones.
Year(s) Of Engagement Activity 2019
 
Description IOM3 Sir Henry Bessemer Master class 2019 - Invited lecture 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Invited lecture entitled "Are you ready for Industry 4.0?", which included group discussions around Industry 4.0 and the steel industry with focus on steel in the digital world and what a digitalized UK steel supply chain could look like.
Year(s) Of Engagement Activity 2019
 
Description IOM3 interview published on IO3M online magazine 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The fellowship research activity was featured on the IOM3 online magazine. The interview appeared online in Dec 2018.
Year(s) Of Engagement Activity 2018
URL https://www.iom3.org/materials-world-magazine/news/2018/dec/10/pursuing-smart-factory
 
Description International Women in Engineering Day Building Smart Things Blog 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Blog of my personal journey to becoming Associate Professor and EPSRC Research Fellow.
Year(s) Of Engagement Activity 2019
URL http://engineeringimpact.co.uk/impact_post/international-women-in-engineering-day-building-smart-thi...
 
Description Invited lecture at the Tata MAGNET Symposium 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited lecture on Industry 4.0 at the MAGNET Symposium. The lecture was well received and generated many questions and interest around the topic. I was asked to attend future events.
Year(s) Of Engagement Activity 2020
 
Description Knowledge Exchange Workshop entitled "Digital Transformation in Manufacturing & Beyond" - 30th October 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact This workshop at Swansea University brought academic and industrial participants together to discuss opportunities and barriers to the adoption of AI-based solutions in manufacturing and its social and economic implications. Participants undertook interactive activities to manually learn and reflect on the following topics:
• State of the art and current solutions for AI-driven manufacturing
• Specific needs of industry for deployment of AI systems in manufacturing
• Human-centric AI in Smart Factories
• Routes to lower adoption barriers
• Industry and academia collaboration to co-create solutions and innovate by building trustworthy AI systems

A panel discussion with Q&A session was followed by two use case scenario based-tasks. The outcome of the workshop was a challenge briefing document, including the group discussion activities, which was circulated to participants to highlight challenges, expertise and resources needed in three main areas: i) machine learning; ii) making a business case; iii) human factors: iv) data. The report has been circulated to participating companies with the view of developing future collaboration in these thematic areas.
Year(s) Of Engagement Activity 2019
 
Description The Times Future Manufacturing supplement 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Dr Giannetti's comments have been included in the Times "Raconteur" supplement (19/02/19) on the Future of Manufacturing. The piece is on page 3 (https://www.raconteur.net/manufacturing/hardware-software-manufacturing) and it asks whether manufacturers should make investments in hardware or software technologies to remain competitive. The Times is a prestigious outlet which reaches around 1 million people a day - print and online combined so the piece has a national and international reach.
Year(s) Of Engagement Activity 2019
URL https://www.raconteur.net/manufacturing/hardware-software-manufacturing