Identifying User Requirements for Digitally Enabling Off-Site Fabrication with Mixed Reality and Gamification

In off-site manufacturing, the bespoke nature of components' design for civil and building infrastructure leads to complex configurations that are difficult to manufacture purely with automation equipment. Labour is heavily involved in most tasks, using tasks and processes very similar to traditional, on site construction. This leads to minimal productivity benefits from producing such components off site. The advent of mixed reality systems, such as HoloLens 2 and its integration with a hardhat plus software by Trimble, creates an opportunity to address this issue off site by completely rethinking the worker tasks and processes to achieve substantial productivity and waste reduction benefits. The project objectives are to leverage point cloud, image and system positioning data produced by mixed reality systems to automatically register virtual components on the build location, determine the optimal build sequence, guide the worker to complete the build, and perform real time dimensional control to self-certify the quality of the build. The vision is to build a cyber-physical system that interacts with the worker.
The PhD project will expect from the student to: a) develop an automatic registration method for registering the design model to the work surface, allowing for manual adjustments; b) develop a near real time component capture method, leveraging the presence of the design model to create an initial hypothesis of the objects and matching the real world captured geometry with the design model in the presence of clutter and spatial errors; c) exploit generative adversarial networks to learn the assembly process from the design model alone; and d) develop a gamification + inspection prototype for an assembly case study for effective worker guidance. As a starting point, the MRes component will involve an extensive end-user requirements analysis through consultations with the FIBE2 partners to map the key activities of the use case and their constraints, along with an initial attempt at optimising build sequences.
The expected outcomes of the project are a) Methods for assembly registration, component capture, gamification and inspection; b) a proof-of-concept prototype; c) publications and presentations at conferences; and d) an exploitation plan. This research will be the first to disrupt off site manufacturing with model-based assembly, real time guidance and progressive, automated inspection. It will also produce a working prototype.

The primary impact of the FIBE2 CDT will be the benefit to society that will accrue from the transformative effect that FIBE2 graduates will have upon current and future infrastructure. The current FIBE CDT has already demonstrated significant impact and FIBE2 will extend this substantially and with particular focus on infrastructure resilience. There will be further impacts across academic research, postgraduate teaching, industry-academia partnering and wider society. Our CDT students are excellent ambassadors and their skills and career trajectories are inspirational. Their outputs so far include >40 journal and conference papers, contributions to a CIRIA report, a book chapter and >15 prizes (e.g. Cambridge Carbon Challenge, EPSRC Doctoral Prizes, best presentation awards). Our students' outreach activities have had far reaching impacts including: Science Festival activities and engineering workshops for school girls. Our innovative CDT training approaches have shifted the culture and priorities in academia and industry towards co-creation for innovation. Our FIBE CDT features in the EPSRC document 'Building Skills for a Prosperous Nation'. Our attention to E&D has resulted in 50% female students with the inspirational ethos attracting students from wide ranging educational backgrounds.

FIBE2 CDT will build on this momentum and expand the scope and reach of our impact. We will capitalise on our major research and training initiatives and strategic collaborations within academia, industry and government to train future infrastructure leaders to address UK and global challenges and this will have direct and significant technical, economic and social impacts for UK infrastructure, its associated stakeholders and civil society at large.

As well as the creation of cohorts of highly skilled research cohorts with cross-disciplinary technical skills, further specific impacts include:

-a transformational cross-disciplinary graduate training and research approach in infrastructure with depth and breadth.

-new forms of Industry-University partnerships. Co-creation with industry of our training and research initiatives has already led to new forms of partnerships such as the I+ scheme, and FIBE2 will further extend this with the 'employer model' variant and others.

-skilled research-minded challenge-focused graduates for UK employers who will derive significant benefit from employing them as catalysts for enterprise, knowledge exchange and innovation, and thus to business growth opportunities.

-enhanced global competitiveness for industrial partners. With our extensive network of 27 industry partners from across all infrastructure sectors who will actively shape the centre with us, we will deliver significant impact and will embrace the cross-disciplinary research emergeing from the CDT to gain competitive advantage.

-support for policy makers at the highest levels of national and local government. The research outcomes and graduates will contribute to an evidence-based foundation for improved decision-making for the efficient management, maintenance and design of infrastructure.

-world-class research outcomes that address national needs, via the direct engagement of our key industrial partners. Other academic institutions will benefit from working with the Centre to collectively advance knowledge.

-wider professional engagement via the creation of powerful informal professional networks between researchers, practitioners, CDT alumni and CDT students, working nationally and internationally, including some hosted by FIBE2 CDT industry partners.

-future generations of infrastructure professional inspired by the FIBE2 CDT's outreach activities whereby pupils, teachers and parents gain insight into the importance of infrastructure engineering.

-the generation of public awareness of the importance of a resilient infrastructure to address inevitable and often unexpected challenges.