Number Understanding Modelling in Behavioural Embodied Robotic Systems

Lead Research Organisation: Sheffield Hallam University
Department Name: College of Business, Technology & Eng

Abstract

Mathematical competence can endow robots of the necessary capability for abstract and symbolic processing, which is required for improving their cognitive performance and their social interaction with human beings. But so far, only few attempts have been made to apply mathematical cognition in robots.

The objective of the NUMBERS project is to construct a novel artificial cognitive model of mathematical cognition by imitating human like learning approaches for developing number understanding. This will provide a novel tool for developmental psychology and neuroscience research on the development of mathematical abilities in children.

The objective will be achieved through a highly interdisciplinary research program that will take advantage of the collaboration of leading academics in the fields involved, prof. Cangelosi (Cognitive Developmental Robotics), Plymouth University, and prof. McClelland (Computational Psychology), Stanford University, USA, and the technical support of an industrial partner (NVIDIA Corporation, USA and UK).

The NUMBERS' research activities will exploit the cutting-edge facilities offered by Sheffield Robotics, a joint initiative between the University of Sheffield and Sheffield Hallam University, which will host the project. Indeed, the model will be integrated with one of the most advanced child-like robotic platform (the "iCub") and, therefore, validated through realistic experiments, resembling scientific experiments of mathematical cognition in children.

The validation of the novel model of the numerical cognition in interactive robotic experiments will constitute a proof of concept of the enhanced capabilities offered by a modular approach to bio-inspired artificial intelligence architectures. Furthermore, an optimised implementation for mobile devices will be realised in order to downsize space and power requirements for the computation, increasing application opportunities.

This foundational research will provide the methodological basis and cognitively plausible engineering principles for the next generation of socially interactive robots, mimicking advanced capabilities of the human intelligence for real understanding and interaction with the external world. Results will help the design of more efficient cognitive robotic systems capable of learning abstract symbolic number processing in a more flexible and ecological manner.

The human-like learning and interaction are characteristics that might allow people to more easily identify the desired social overture that the robot is making, or facilitate the transfer of skills learned in human-human interactions to human-robot encounters. This envisioned humanization will positively affect the acceptance of robots in social environments, as they will be perceived as less dangerous, increasing the socio-economic applications of future robots that can take on tasks once thought too delicate or uneconomical to automate. This is particularly relevant in the fields of social care, companionship, therapy, domestic assistance, entertainment, and education.

Planned Impact

The NUMBERS project is based on a highly interdisciplinary approach and has an important scientific and technological bearing that will have a significant impact in the technological fields (e.g. service robotics and HRI for interactive systems, GPU accelerated parallel computing, robotics engineering) covered by the project.

From a socio-economic point of view, the technological advances expected will facilitate the design of novel control systems suitable for a wide range of commercial interactive system applications (e.g. social robots, smart intelligent devices, speech interfaces, autonomous and intelligent-assisted transport vehicles). Indeed, the NUMBERS project aims to be one seed for future UK industrial leadership in the emerging social robotics industry, as human-like cognition and interaction will drastically increase the acceptance of robots in social environments, widening future applications, such as care for elderly, child education, entertainment, and domestic use.
In particular, in the field of social care and innovation, the project foresees synergies with the H2020 MSCA CARER-AID project, which has the same host institution (SHU) and supervisor (PI). These synergies will allow extending the contribution to the assessment and therapeutic technologies for cognitively disabled children.

From an industrial point of view, the project will also contribute towards widening the application of deep learning technologies that are currently one of the most vital pieces of work in computing industry of today. Indeed, top industry players like NVIDIA, Google, and Microsoft are strongly investing in expanding this technology for their future products. The sponsorship for the NUMBERS project by the NVIDIA Corporation is evidence of the interest and offers a great opportunity for collaborating towards the development of a profitable technology for the future robotics applications in social environments. In addition, embedded microchip design houses, such as ARM Ltd, can profit from the targeted optimisation of the model for portable devices, which can be more likely to be integrated into mobile socially interactive robotic platforms.

For knowledge transfer into industries, the PI will communicate the project ideas and results to the NVIDIA GTC conferences which can allow for extending the project's impact by interacting with software developers, researchers, and technologists from some of the top companies, universities, research firms, and government agencies from around the world. The PI will organise an Industrial Open House, inviting delegates from NVIDIA Corporation, ARM Ltd and UK robotics companies, e.g. RU Robots Ltd, AA Robotics, Engineered Arts Ltd, and Shadow Robotics.

Public Engagement is one of the objectives of the present project, as more information is needed to overcome the scepticism that currently regards the use of robots in the social context. This will be achieved taking all the available opportunities, such as SHU open days, Sheffield Robotics outreach events and during the UK robotics week.

A dedicated project website will be accelerating dissemination and communication away from academic publications and public events to all relevant parties. An extensive online resource will include videos of the experiments and demonstrators of the advanced robot behaviours, presentations, highlights from publications. The software developed within the project will be available open source through the project website.

Publications

10 25 50
publication icon
Conti D (2020) "Robot, tell me a tale!" A social robot as tool for teachers in kindergarten in Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems

publication icon
Conti D (2021) An empirical study on integrating a small humanoid robot to support the therapy of children with Autism Spectrum Disorder and Intellectual Disability in Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems

publication icon
Di Nuovo A (2019) Assessment of Cognitive skills via Human-robot Interaction and Cloud Computing in Journal of Bionic Engineering

publication icon
Di Nuovo A (2021) Abstract Concept Learning in Cognitive Robots in Current Robotics Reports

publication icon
Di Nuovo A (2019) Developing the knowledge of number digits in a child-like robot in Nature Machine Intelligence

publication icon
Wang N (2021) A Framework of Hybrid Force/Motion Skills Learning for Robots in IEEE Transactions on Cognitive and Developmental Systems

 
Description The project delivered a model that replicates in robotics the children learning behaviour for understanding digits and numbers. Indeed, the research evidenced similarities between children and robotic embodiment in the early development of numerical cognition.
Exploitation Route These developmental neurorobotics models show the way to build open-ended minds for interactive robots, which will carry out more realistic services by autonomously developing the required behavioural and cognitive capabilities via direct interaction with human teachers and the environment.
Cross-disciplinary applications of these developmental neurorobotics models can be found in life sciences, e.g. developmental psychology, neuroscience, and healthcare. Indeed, computational models are a powerful tool to quickly generate information and make novel predictions before being tested on humans. Interestingly, models can be lesioned to simulate cognitive dysfunctions, which would allow gaining information for diagnosis and treatment that are difficult to discover otherwise.
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Healthcare,Other

URL https://www.researchgate.net/project/Number-Understanding-Modelling-in-Behavioural-Embodied-Robotic-Systems
 
Description (PERSEO) - European Training Network on PErsonalized Robotics as SErvice Oriented applications
Amount € 4,096,839 (EUR)
Funding ID 955778 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 01/2021 
End 12/2024
 
Description Autonomous Learning and Development in Embodied Neuromorphic Systems (ALDENS)
Amount £202,143 (GBP)
Funding ID EP/X018733/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2022 
End 09/2024
 
Title Number Understanding Modelling in Behavioural Embodied Robotic Systems (NUMBERS). SHU Research Data Archive (SHURDA) 
Description Deep Learning; Cognitive Developmental Robotics; Number Cognition; Embodied Intelligence Data may be available from external sources: https://github.com/EPSRC-NUMBERS 
Type Of Material Computer model/algorithm 
Year Produced 2019 
Provided To Others? Yes  
Impact none 
URL http://github.com/EPSRC-NUMBERS
 
Description NVIDIA Academic Collaboration 
Organisation NVIDIA
Country Global 
Sector Private 
PI Contribution Demonstrate the potential use of NVIDIA hardware in the development of deep learning embodied models of the numerical cognition in robots.
Collaborator Contribution NVIDIA, as part of the Academic Programs, selected Dr. Di Nuovo's research project and donated to Sheffield Hallam University: (2) NVIDIA Tesla K40 and (1) NVIDIA Titan X graphic cards. The donation aims to empower Dr. Di Nuovo's development of the computational models.
Impact Presentation at the NVIDIA GTC Europe 2018
Start Year 2016
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Manchester
Department School of Computer Science
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Plymouth
Department Centre for Robotics and Neural Systems (CNRS)
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Article for the Conversation 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact An article for the conversation which has been read by 26265 people (statistics from the website on 01/03/2019), and reprinted by the Independent.
Year(s) Of Engagement Activity 2018
URL https://theconversation.com/robot-carers-could-help-lonely-seniors-theyre-cheering-humans-up-already...
 
Description Invited talk at the NVIDIA European GPU Technology Conference (GTC) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact NVIDIA's GPU Technology Conference (GTC) is a global conference series providing training, insights, and direct access to experts on the hottest topics in computing today.
Year(s) Of Engagement Activity 2018
URL https://tinyurl.com/y974ucy2
 
Description L3C AI in ActIon 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact AI/ML is being applied across multiple industries to provide better insights and enhance decision making. L3C and our sponsors and partners IBM, E2Exchange, ArrowECS have joined together to bring you real examples of where AI/ML is being applied in your industry.

The Royal Institution was founded in 1799 with the aim of introducing new technologies and teaching science. With this heritage in mind we have chosen this esteemed venue for our unique event to show you how AI/ML technology is being used today.

Join us to hear from leaders and innovators showing how AI/ML is being deployed across Insurance, Banking, Retail, Healthcare, Media and HR as well as points of view from leading technology companies such as IBM and Nvidia.

Those with responsibility for ML/AI projects in medium and large enterprises, Heads of ML/AI, BigData, Analytics, Data Scientists as well as those leading or involved in business transformation projects will gain real insight to what is really happening in industries today. Smaller companies looking to disrupt and gain advantage or who address a specific industry issue will gain increased awareness of the state of play.

The day will be a mix of thought provoking and challenging discussions with coffee/networking breaks and lunch culminating in an end of day drinks reception.
Year(s) Of Engagement Activity 2018
URL https://l3c.cloud/l3c-ai-in-action/
 
Description Lectures @ PERSEO Summer School 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The 2022 Summer School on Service-Based and Cloud Robotics was organised by the PERSEO ETN European Doctoral Training Network on Personalised Robotics. The school will consist of lectures from the international experts in the field. Lectures sparked questions and discussions increasing the knowledge and interest of the PG students in the topics.
Year(s) Of Engagement Activity 2022
URL https://perseo-summer-school.github.io/
 
Description Presentation at the HONDA HARU Fest 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Haru Fest 2022
Recently Tokyo hosted to the inaugural Haru Fest 2022! Attended by over 100 academics, engineers, and writers from around the world, Haru Fest was an opportunity not only to celebrate the social robots of Honda Research Institute Japan's, but also for the academics and professionals who designed Haru to share their work and discuss the future of social robots.
Haru Fest's theme was empathy, integration, and diversity, and it focused on issues in empathy, bond and trust with social robots and their integration into society. It took place over 2 days jam-packed with inspiring invited talks, insightful panel sessions, and informative flash talks featuring visionaries from industry and academia.
Last but not least, Haru Fest featured awe-inspiring demonstrations of capabilities from both Honda's pioneering social robot, ASIMO, and from Haru itself. Attendees left feeling knowledgeable about, connected to, and optimistic about Haru's social robotics community.
Year(s) Of Engagement Activity 2022
URL https://mypersonalrobots.org/events/2022/10/29/haru-fest-2022-a-gathering-of-the-tribes
 
Description Presentation for the local branch of the British Computer Society (open to public) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Presentation at the local British Computer Society branch, title: Programming robots that care for humans
Date/Time:
Thursday 19 April 2018, 5.30pm - 7.30pm

Venue:
Room 9130, Cantor Building, City Campus, Sheffield Hallam University S1 2ND | Map
Buffet to be served in Room 9020a

The event will start with light refreshments at 5.30pm with the talk commencing at 6.30pm and concluding at 7.30pm

Details:

Recent advances in computing technology and novel artificial intelligence algorithms enable investigation on social applications of robotics, such as the education of children and the care of the elderly. The investigation has the objective to create robots capable of human-like social interaction while exploiting their computational capability for data collection and processing.

Current Social Robots are still very far from those envisioned in science fiction novels, indeed there are only few success cases outside the University labs. Nevertheless, the scientific literature shows the tremendous potential of robotic systems to provide individualised assistance to some classes of people, e.g. children with autism, elderly with dementia.

The talk will present examples from current research projects of Sheffield Robotics at Sheffield Hallam University in the area of socially assistive robotics.

This will show the current state-of-the-art, increase the awareness of benefits and limitation, and discuss the potential breakthroughs and implications for professionals in computing and the general population.

The talk will emphasise responsible user-centred research with the aim to empower people rather than substitute them.
Year(s) Of Engagement Activity 2018
URL https://www.bcs.org/content/ConWebDoc/59182
 
Description Radio Interview for the BBC World program Sounds on How technology can help look after an ageing population 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by the BBC World program the Sounds on How technology can help look after an ageing population.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/sounds/play/w3cswgwk
 
Description Seminar at the TRAINCRASE Workshop in Manchester 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact Talk about Modelling abstract and numerical concepts sparkled great interest and many questions from the audience.
Year(s) Of Engagement Activity 2022
 
Description Talk Social Applications of Multimodal Cognitive Robots 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This workshop shared synergetic contribution of different research areas e.g., Artificial Intelligence (AI), Internet-of-Things (IoT), Robotics, and Social Science to develop innovative and effective research and technologies for the ageing society and assistance in general.
Year(s) Of Engagement Activity 2022