Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications (Jarvis)

To strengthen the competitiveness of the EU industry, there is an increasing need for flexible production, exploiting the capabilities of both the machinery and human workforce. The growing market of robots has reacted, adapting to the changing situation by providing collaborative robotic solutions with a variety of characteristicsin terms of payload, type and mobility. A number of technologicalsolutions for perception and safety allow their coexistence with humans. While several research activities have been conducted over the past 5 years on this topic, wider industrial adoption is lagging due to the following restrictions: • Limited cognition and intelligence • Low performance of collaborative operations • Collaboration fluidity is rather low as the operators have to adapt to the particularities of the robots. • Complexity in robot programming which requires the involvement of skilled engineers, does not provide flexibility in the execution phase, and does not benefit from the tacit knowledge of experienced operators. Thus, despite the fact the mechatronics are quite advanced, a perfect matching of mechanics and control is required to create robots that advance from repetitive and precision oriented tasks to becoming intelligent and helpful co-workers. Proper human robot interaction is the most important ground to be covered. Thus, JARVIS aims to develop a reusable set of tools that enable AI driven multimodal means of interaction: a) involving interfaces for physical and remote information exchange, robot control and programming, b) providing social skills to a variety of robots to achieve seamless user-centric interaction that extends human ability for complex tasks and c) demonstrating scalability of application and ability to achieve economies at scale.