Digitally Enabling Off-Site Fabrication with Mixed Reality and Gamification

Lead Research Organisation: University of Cambridge
Department Name: Engineering


In off-site manufacturing, the bespoke nature of components' design for civil and building infrastructure leads to complex configurations that are difficult to manufacture purely with automation equipment. Labour is heavily involved in most tasks, using tasks and processes very similar to traditional, on site construction. This leads to minimal productivity benefits from producing such components off site. The advent of mixed reality systems, such as HoloLens 2 and its integration with a hardhat plus software by Trimble, creates an opportunity to address this issue off site by completely rethinking the worker tasks and processes to achieve substantial productivity and waste reduction benefits. The project objectives are to leverage point cloud, image and system positioning data produced by mixed reality systems to automatically register virtual components on the build location, determine the optimal build sequence, guide the worker to complete the build, and perform real time dimensional control to self-certify the quality of the build. The vision is to build a cyber-physical system that interacts with the worker.
The PhD project will expect from the student to: a) develop an automatic registration method for registering the design model to the work surface, allowing for manual adjustments; b) develop a near real time component capture method, leveraging the presence of the design model to create an initial hypothesis of the objects and matching the real world captured geometry with the design model in the presence of clutter and spatial errors; c) exploit generative adversarial networks to learn the assembly process from the design model alone; and d) develop a gamification + inspection prototype for an assembly case study for effective worker guidance. As a starting point, the MRes component will involve an extensive end-user requirements analysis through consultations with the FIBE2 partners to map the key activities of the use case and their constraints, along with an initial attempt at optimising build sequences.
The expected outcomes of the project are a) Methods for assembly registration, component capture, gamification and inspection; b) a proof-of-concept prototype; c) publications and presentations at conferences; and d) an exploitation plan. This research will be the first to disrupt off site manufacturing with model-based assembly, real time guidance and progressive, automated inspection. It will also produce a working prototype.


10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/S02302X/1 30/09/2019 30/03/2028
2439669 Studentship EP/S02302X/1 30/09/2020 29/09/2024 K-M Klehe White