Number Understanding Modelling in Behavioural Embodied Robotic Systems

Lead Research Organisation: Sheffield Hallam University
Department Name: College of Business, Technology & Eng

Abstract

Mathematical competence can endow robots of the necessary capability for abstract and symbolic processing, which is required for improving their cognitive performance and their social interaction with human beings. But so far, only few attempts have been made to apply mathematical cognition in robots.

The objective of the NUMBERS project is to construct a novel artificial cognitive model of mathematical cognition by imitating human like learning approaches for developing number understanding. This will provide a novel tool for developmental psychology and neuroscience research on the development of mathematical abilities in children.

The objective will be achieved through a highly interdisciplinary research program that will take advantage of the collaboration of leading academics in the fields involved, prof. Cangelosi (Cognitive Developmental Robotics), Plymouth University, and prof. McClelland (Computational Psychology), Stanford University, USA, and the technical support of an industrial partner (NVIDIA Corporation, USA and UK).

The NUMBERS' research activities will exploit the cutting-edge facilities offered by Sheffield Robotics, a joint initiative between the University of Sheffield and Sheffield Hallam University, which will host the project. Indeed, the model will be integrated with one of the most advanced child-like robotic platform (the "iCub") and, therefore, validated through realistic experiments, resembling scientific experiments of mathematical cognition in children.

The validation of the novel model of the numerical cognition in interactive robotic experiments will constitute a proof of concept of the enhanced capabilities offered by a modular approach to bio-inspired artificial intelligence architectures. Furthermore, an optimised implementation for mobile devices will be realised in order to downsize space and power requirements for the computation, increasing application opportunities.

This foundational research will provide the methodological basis and cognitively plausible engineering principles for the next generation of socially interactive robots, mimicking advanced capabilities of the human intelligence for real understanding and interaction with the external world. Results will help the design of more efficient cognitive robotic systems capable of learning abstract symbolic number processing in a more flexible and ecological manner.

The human-like learning and interaction are characteristics that might allow people to more easily identify the desired social overture that the robot is making, or facilitate the transfer of skills learned in human-human interactions to human-robot encounters. This envisioned humanization will positively affect the acceptance of robots in social environments, as they will be perceived as less dangerous, increasing the socio-economic applications of future robots that can take on tasks once thought too delicate or uneconomical to automate. This is particularly relevant in the fields of social care, companionship, therapy, domestic assistance, entertainment, and education.

Planned Impact

The NUMBERS project is based on a highly interdisciplinary approach and has an important scientific and technological bearing that will have a significant impact in the technological fields (e.g. service robotics and HRI for interactive systems, GPU accelerated parallel computing, robotics engineering) covered by the project.

From a socio-economic point of view, the technological advances expected will facilitate the design of novel control systems suitable for a wide range of commercial interactive system applications (e.g. social robots, smart intelligent devices, speech interfaces, autonomous and intelligent-assisted transport vehicles). Indeed, the NUMBERS project aims to be one seed for future UK industrial leadership in the emerging social robotics industry, as human-like cognition and interaction will drastically increase the acceptance of robots in social environments, widening future applications, such as care for elderly, child education, entertainment, and domestic use.
In particular, in the field of social care and innovation, the project foresees synergies with the H2020 MSCA CARER-AID project, which has the same host institution (SHU) and supervisor (PI). These synergies will allow extending the contribution to the assessment and therapeutic technologies for cognitively disabled children.

From an industrial point of view, the project will also contribute towards widening the application of deep learning technologies that are currently one of the most vital pieces of work in computing industry of today. Indeed, top industry players like NVIDIA, Google, and Microsoft are strongly investing in expanding this technology for their future products. The sponsorship for the NUMBERS project by the NVIDIA Corporation is evidence of the interest and offers a great opportunity for collaborating towards the development of a profitable technology for the future robotics applications in social environments. In addition, embedded microchip design houses, such as ARM Ltd, can profit from the targeted optimisation of the model for portable devices, which can be more likely to be integrated into mobile socially interactive robotic platforms.

For knowledge transfer into industries, the PI will communicate the project ideas and results to the NVIDIA GTC conferences which can allow for extending the project's impact by interacting with software developers, researchers, and technologists from some of the top companies, universities, research firms, and government agencies from around the world. The PI will organise an Industrial Open House, inviting delegates from NVIDIA Corporation, ARM Ltd and UK robotics companies, e.g. RU Robots Ltd, AA Robotics, Engineered Arts Ltd, and Shadow Robotics.

Public Engagement is one of the objectives of the present project, as more information is needed to overcome the scepticism that currently regards the use of robots in the social context. This will be achieved taking all the available opportunities, such as SHU open days, Sheffield Robotics outreach events and during the UK robotics week.

A dedicated project website will be accelerating dissemination and communication away from academic publications and public events to all relevant parties. An extensive online resource will include videos of the experiments and demonstrators of the advanced robot behaviours, presentations, highlights from publications. The software developed within the project will be available open source through the project website.
 
Description The project delivered a model that replicates in robotics the children learning behaviour for understanding digits and numbers. Indeed, the research evidenced similarities between children and robotic embodiment in the early development of numerical cognition.
Exploitation Route These developmental neurorobotics models show the way to build open-ended minds for interactive robots, which will carry out more realistic services by autonomously developing the required behavioural and cognitive capabilities via direct interaction with human teachers and the environment.
Cross-disciplinary applications of these developmental neurorobotics models can be found in life sciences, e.g. developmental psychology, neuroscience, and healthcare. Indeed, computational models are a powerful tool to quickly generate information and make novel predictions before being tested on humans. Interestingly, models can be lesioned to simulate cognitive dysfunctions, which would allow gaining information for diagnosis and treatment that are difficult to discover otherwise.
Sectors Communities and Social Services/Policy

Digital/Communication/Information Technologies (including Software)

Education

Healthcare

Other

URL https://www.researchgate.net/project/Number-Understanding-Modelling-in-Behavioural-Embodied-Robotic-Systems
 
Description The dissemination of the research findings from this project extended beyond traditional academic channels to reach a diverse audience through various mediums. Open events, including live demonstrations of the AI-powered robotic technologies, public talks, and lectures, provided platforms for engaging with industry and the wider public directly, allowing for interactive discussions and the sharing of insights in accessible formats. These events not only served to inform and educate attendees but also fostered dialogue around the use of artificial intelligence in everyday life. In addition to in-person events, the research findings were also shared extensively through interviews, magazine articles, and social media platforms, LinkedIn and Twitter (now X), to reach a broader audience globally (over 1,000 followers on social media), thereby amplifying the dissemination efforts and extending the reach of the research to diverse online communities. As result of the public engagment (in person and online), there is evidence of increase in public awareness of the cognitive developmental robotics research potential to make economic and societal changes, also an improved attitude toward AI and Robotics. Furthermore, the research findings played a pivotal role in driving tangible education outcomes, such as the development and implementation of a module on Cognitive Robotics and Systems for the new Masters in AI course at Sheffield Hallam University. The results of the project were also presented at summer schools and other educational events.
First Year Of Impact 2018
Sector Digital/Communication/Information Technologies (including Software),Education
Impact Types Cultural

Societal

 
Description Contribution to the development of a new Cognitive Robotics and Systems module for the Masters in Artificial Intelligence at Sheffield Hallam University
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact knowledge of advanced cognitive robotics solutions
 
Description influence on the European doctoral consortium PERSEO
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact The personal robotics domain is raising new challenges concerning the need for robot behaviour with a high level of personalisation with respect to each user's needs and preferences.
URL https://www.perseo.eu/
 
Description (PERSEO) - European Training Network on PErsonalized Robotics as SErvice Oriented applications
Amount € 4,096,839 (EUR)
Funding ID 955778 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 01/2021 
End 12/2024
 
Description Autonomous Learning and Development in Embodied Neuromorphic Systems (ALDENS)
Amount £202,143 (GBP)
Funding ID EP/X018733/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2023 
End 09/2025
 
Description I'M-ACTIVE : Intelligent Multimodal Assessment and Coaching Through Identification of Vulnerabilities in older pEople
Amount £400,481 (GBP)
Funding ID EP/W031809/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2023 
End 06/2024
 
Description Performance in Robots Interaction via Mental Imagery - Call HORIZON-CL4-2022-DIGITAL-EMERGING-02-06 - Pushing the limit of physical intelligence and performance (RIA)
Amount € 7,300,000 (EUR)
Funding ID 101120727 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 11/2023 
End 12/2027
 
Title Number Understanding Modelling in Behavioural Embodied Robotic Systems (NUMBERS). SHU Research Data Archive (SHURDA) 
Description Deep Learning; Cognitive Developmental Robotics; Number Cognition; Embodied Intelligence Data may be available from external sources: https://github.com/EPSRC-NUMBERS 
Type Of Material Computer model/algorithm 
Year Produced 2019 
Provided To Others? Yes  
Impact none 
URL http://github.com/EPSRC-NUMBERS
 
Description NVIDIA Academic Collaboration 
Organisation NVIDIA
Country Global 
Sector Private 
PI Contribution Demonstrate the potential use of NVIDIA hardware in the development of deep learning embodied models of the numerical cognition in robots.
Collaborator Contribution NVIDIA, as part of the Academic Programs, selected Dr. Di Nuovo's research project and donated to Sheffield Hallam University: (2) NVIDIA Tesla K40 and (1) NVIDIA Titan X graphic cards. The donation aims to empower Dr. Di Nuovo's development of the computational models.
Impact Presentation at the NVIDIA GTC Europe 2018
Start Year 2016
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Manchester
Department School of Computer Science
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Plymouth
Department Centre for Robotics and Neural Systems (CNRS)
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Article for the Conversation 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact An article for the conversation which has been read by 26265 people (statistics from the website on 01/03/2019), and reprinted by the Independent.
Year(s) Of Engagement Activity 2018
URL https://theconversation.com/robot-carers-could-help-lonely-seniors-theyre-cheering-humans-up-already...
 
Description Centre for Artificial Intelligence and RObotics (CAIRO) Invited Speakers Series - Technical University of Applied Sciences Würzburg-Schweinfurt 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Invited talk "How can bio-inspired robotic platforms help develop new artificial intelligence (AI) services to benefit society?" given at the Centre for Artificial Intelligence and RObotics (CAIRO) Invited Speakers Series - Technical University of Applied Sciences Würzburg-Schweinfurt
The talk contributed to strenghten the collaboration between CAIRO and the SHU Department of Computing, including an agreement to exchange staff and postgraduate students, with a particular focus on the topics covered by the talk.
Year(s) Of Engagement Activity 2023
URL https://www.thws.de/forschung/institute/cairo/invited-speakers-series/
 
Description Exhibition at the IEEE International Conference of Robotics and Automation (ICRA) 2023 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact ICRA is the largest robotics conference. 6000 attendees overall from academia, industry, media and the public.
Year(s) Of Engagement Activity 2023
URL https://www.icra2023.org/
 
Description Invited talk at the NVIDIA European GPU Technology Conference (GTC) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact NVIDIA's GPU Technology Conference (GTC) is a global conference series providing training, insights, and direct access to experts on the hottest topics in computing today.
Year(s) Of Engagement Activity 2018
URL https://tinyurl.com/y974ucy2
 
Description L3C AI in ActIon 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact AI/ML is being applied across multiple industries to provide better insights and enhance decision making. L3C and our sponsors and partners IBM, E2Exchange, ArrowECS have joined together to bring you real examples of where AI/ML is being applied in your industry.

The Royal Institution was founded in 1799 with the aim of introducing new technologies and teaching science. With this heritage in mind we have chosen this esteemed venue for our unique event to show you how AI/ML technology is being used today.

Join us to hear from leaders and innovators showing how AI/ML is being deployed across Insurance, Banking, Retail, Healthcare, Media and HR as well as points of view from leading technology companies such as IBM and Nvidia.

Those with responsibility for ML/AI projects in medium and large enterprises, Heads of ML/AI, BigData, Analytics, Data Scientists as well as those leading or involved in business transformation projects will gain real insight to what is really happening in industries today. Smaller companies looking to disrupt and gain advantage or who address a specific industry issue will gain increased awareness of the state of play.

The day will be a mix of thought provoking and challenging discussions with coffee/networking breaks and lunch culminating in an end of day drinks reception.
Year(s) Of Engagement Activity 2018
URL https://l3c.cloud/l3c-ai-in-action/
 
Description Lecture for the local branch of the British Computer Society (open to public) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Presentation at the local British Computer Society branch, title: Programming robots that care for humans
Date/Time:
Thursday 19 April 2018, 5.30pm - 7.30pm

Venue:
Room 9130, Cantor Building, City Campus, Sheffield Hallam University S1 2ND | Map
Buffet to be served in Room 9020a

The event will start with light refreshments at 5.30pm with the talk commencing at 6.30pm and concluding at 7.30pm

Details:

Recent advances in computing technology and novel artificial intelligence algorithms enable investigation on social applications of robotics, such as the education of children and the care of the elderly. The investigation has the objective to create robots capable of human-like social interaction while exploiting their computational capability for data collection and processing.

Current Social Robots are still very far from those envisioned in science fiction novels, indeed there are only few success cases outside the University labs. Nevertheless, the scientific literature shows the tremendous potential of robotic systems to provide individualised assistance to some classes of people, e.g. children with autism, elderly with dementia.

The talk will present examples from current research projects of Sheffield Robotics at Sheffield Hallam University in the area of socially assistive robotics.

This will show the current state-of-the-art, increase the awareness of benefits and limitation, and discuss the potential breakthroughs and implications for professionals in computing and the general population.

The talk will emphasise responsible user-centred research with the aim to empower people rather than substitute them.
Year(s) Of Engagement Activity 2018
URL https://www.bcs.org/content/ConWebDoc/59182
 
Description Lectures @ PERSEO Summer School 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The 2022 Summer School on Service-Based and Cloud Robotics was organised by the PERSEO ETN European Doctoral Training Network on Personalised Robotics. The school will consist of lectures from the international experts in the field. Lectures sparked questions and discussions increasing the knowledge and interest of the PG students in the topics.
Year(s) Of Engagement Activity 2022
URL https://perseo-summer-school.github.io/
 
Description Presentation at the HONDA HARU Fest 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Haru Fest 2022
Recently Tokyo hosted to the inaugural Haru Fest 2022! Attended by over 100 academics, engineers, and writers from around the world, Haru Fest was an opportunity not only to celebrate the social robots of Honda Research Institute Japan's, but also for the academics and professionals who designed Haru to share their work and discuss the future of social robots.
Haru Fest's theme was empathy, integration, and diversity, and it focused on issues in empathy, bond and trust with social robots and their integration into society. It took place over 2 days jam-packed with inspiring invited talks, insightful panel sessions, and informative flash talks featuring visionaries from industry and academia.
Last but not least, Haru Fest featured awe-inspiring demonstrations of capabilities from both Honda's pioneering social robot, ASIMO, and from Haru itself. Attendees left feeling knowledgeable about, connected to, and optimistic about Haru's social robotics community.
Year(s) Of Engagement Activity 2022
URL https://mypersonalrobots.org/events/2022/10/29/haru-fest-2022-a-gathering-of-the-tribes
 
Description Public lecture 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Professorial inaugural lecture, open and widely advertise to the public. Around 100 participants, including form the public, e.g. 15 registered participants were from the university of third age.
Year(s) Of Engagement Activity 2023
URL https://blog.shu.ac.uk/awrc/2023/05/17/prof-alessandro-di-nuovo-inaugural-lecture/
 
Description Radio Interview for the BBC World program Sounds on How technology can help look after an ageing population 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by the BBC World program the Sounds on How technology can help look after an ageing population.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/sounds/play/w3cswgwk
 
Description Research projects presentation at the 7th IEEE UK and Ireland Robotics and Autonomous Systems Conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact The conference attracted over 200 participants of which 118 were postgraduate or undergraduate students, 20 from industry/business.
Year(s) Of Engagement Activity 2024
URL https://www.ieee-ukandireland.org/event/ras-2024-7th-annual-ieee-uk-and-ireland-robotics-and-automat...
 
Description Seminar at the TRAINCRASE Workshop in Manchester 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact Talk about Modelling abstract and numerical concepts sparkled great interest and many questions from the audience.
Year(s) Of Engagement Activity 2022
 
Description Shonan Meeting 177: Natural Interaction with Humanoid Robots 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact The group worked intensively for 4 days and explored possible challenges and jointly developed a research agenda for main research directions in the topic. The meeting establish an international collaboration for follow-up research.
Year(s) Of Engagement Activity 2023
URL https://shonan.nii.ac.jp/seminars/177/
 
Description Talk Social Applications of Multimodal Cognitive Robots 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This workshop shared synergetic contribution of different research areas e.g., Artificial Intelligence (AI), Internet-of-Things (IoT), Robotics, and Social Science to develop innovative and effective research and technologies for the ageing society and assistance in general.
Year(s) Of Engagement Activity 2022
 
Description White Paper on Security and Privacy in Assistive Robotics: Cybersecurity Challenges for Healthcare 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The UK-RAS white papers serve as a basis for discussing the future technological roadmaps, engaging the wider community and stakeholders, as well as policy makers in assessing the potential social, economic and ethical/legal impact of RAS.
This white paper explores the security and privacy needs for Robotics and Autonomous Systems (RAS) working in healthcare. RAS devices in the care domain will enable people a greater degree of independence, with less reliance on other people, and this in turn could enable people to remain longer in their own homes.
Year(s) Of Engagement Activity 2023
URL https://www.ukras.org.uk/publications/white-papers/security-and-privacy-in-assistive-robotics/