Number Understanding Modelling in Behavioural Embodied Robotic Systems

Lead Research Organisation: Sheffield Hallam University
Department Name: Faculty of Arts Computing Eng and Sci

Abstract

Mathematical competence can endow robots of the necessary capability for abstract and symbolic processing, which is required for improving their cognitive performance and their social interaction with human beings. But so far, only few attempts have been made to apply mathematical cognition in robots.

The objective of the NUMBERS project is to construct a novel artificial cognitive model of mathematical cognition by imitating human like learning approaches for developing number understanding. This will provide a novel tool for developmental psychology and neuroscience research on the development of mathematical abilities in children.

The objective will be achieved through a highly interdisciplinary research program that will take advantage of the collaboration of leading academics in the fields involved, prof. Cangelosi (Cognitive Developmental Robotics), Plymouth University, and prof. McClelland (Computational Psychology), Stanford University, USA, and the technical support of an industrial partner (NVIDIA Corporation, USA and UK).

The NUMBERS' research activities will exploit the cutting-edge facilities offered by Sheffield Robotics, a joint initiative between the University of Sheffield and Sheffield Hallam University, which will host the project. Indeed, the model will be integrated with one of the most advanced child-like robotic platform (the "iCub") and, therefore, validated through realistic experiments, resembling scientific experiments of mathematical cognition in children.

The validation of the novel model of the numerical cognition in interactive robotic experiments will constitute a proof of concept of the enhanced capabilities offered by a modular approach to bio-inspired artificial intelligence architectures. Furthermore, an optimised implementation for mobile devices will be realised in order to downsize space and power requirements for the computation, increasing application opportunities.

This foundational research will provide the methodological basis and cognitively plausible engineering principles for the next generation of socially interactive robots, mimicking advanced capabilities of the human intelligence for real understanding and interaction with the external world. Results will help the design of more efficient cognitive robotic systems capable of learning abstract symbolic number processing in a more flexible and ecological manner.

The human-like learning and interaction are characteristics that might allow people to more easily identify the desired social overture that the robot is making, or facilitate the transfer of skills learned in human-human interactions to human-robot encounters. This envisioned humanization will positively affect the acceptance of robots in social environments, as they will be perceived as less dangerous, increasing the socio-economic applications of future robots that can take on tasks once thought too delicate or uneconomical to automate. This is particularly relevant in the fields of social care, companionship, therapy, domestic assistance, entertainment, and education.

Planned Impact

The NUMBERS project is based on a highly interdisciplinary approach and has an important scientific and technological bearing that will have a significant impact in the technological fields (e.g. service robotics and HRI for interactive systems, GPU accelerated parallel computing, robotics engineering) covered by the project.

From a socio-economic point of view, the technological advances expected will facilitate the design of novel control systems suitable for a wide range of commercial interactive system applications (e.g. social robots, smart intelligent devices, speech interfaces, autonomous and intelligent-assisted transport vehicles). Indeed, the NUMBERS project aims to be one seed for future UK industrial leadership in the emerging social robotics industry, as human-like cognition and interaction will drastically increase the acceptance of robots in social environments, widening future applications, such as care for elderly, child education, entertainment, and domestic use.
In particular, in the field of social care and innovation, the project foresees synergies with the H2020 MSCA CARER-AID project, which has the same host institution (SHU) and supervisor (PI). These synergies will allow extending the contribution to the assessment and therapeutic technologies for cognitively disabled children.

From an industrial point of view, the project will also contribute towards widening the application of deep learning technologies that are currently one of the most vital pieces of work in computing industry of today. Indeed, top industry players like NVIDIA, Google, and Microsoft are strongly investing in expanding this technology for their future products. The sponsorship for the NUMBERS project by the NVIDIA Corporation is evidence of the interest and offers a great opportunity for collaborating towards the development of a profitable technology for the future robotics applications in social environments. In addition, embedded microchip design houses, such as ARM Ltd, can profit from the targeted optimisation of the model for portable devices, which can be more likely to be integrated into mobile socially interactive robotic platforms.

For knowledge transfer into industries, the PI will communicate the project ideas and results to the NVIDIA GTC conferences which can allow for extending the project's impact by interacting with software developers, researchers, and technologists from some of the top companies, universities, research firms, and government agencies from around the world. The PI will organise an Industrial Open House, inviting delegates from NVIDIA Corporation, ARM Ltd and UK robotics companies, e.g. RU Robots Ltd, AA Robotics, Engineered Arts Ltd, and Shadow Robotics.

Public Engagement is one of the objectives of the present project, as more information is needed to overcome the scepticism that currently regards the use of robots in the social context. This will be achieved taking all the available opportunities, such as SHU open days, Sheffield Robotics outreach events and during the UK robotics week.

A dedicated project website will be accelerating dissemination and communication away from academic publications and public events to all relevant parties. An extensive online resource will include videos of the experiments and demonstrators of the advanced robot behaviours, presentations, highlights from publications. The software developed within the project will be available open source through the project website.
 
Description The project presented computational models that show the close relationship of numerical cognition with the human body, in particular the fingers. This provides a strong motivation for the use of an interdisciplinary approach known as Cognitive Developmental Robotics for the computational modelling of such fundamental component of human intelligence.
Exploitation Route deploying robots in the classroom to assist the teacher in the mathematical education of children. These teaching assistants could be peers, i.e. capable to mimic the behaviours of children when learning mathematics. These robots can lead educational activities in form of a game, during which they interact with speech and gestures to guide the learner through arithmetic procedures or and prompt the children to identifying errors in the robot behaviours.
Sectors Education,Healthcare

URL http://blogs.shu.ac.uk/shurobotics/category/numbers/
 
Title Number Understanding Modelling in Behavioural Embodied Robotic Systems (NUMBERS). SHU Research Data Archive (SHURDA) 
Description Deep Learning; Cognitive Developmental Robotics; Number Cognition; Embodied Intelligence Data may be available from external sources: https://github.com/EPSRC-NUMBERS 
Type Of Material Computer model/algorithm 
Year Produced 2019 
Provided To Others? Yes  
Impact none 
URL http://github.com/EPSRC-NUMBERS
 
Description NVIDIA Academic Collaboration 
Organisation NVIDIA
Country Global 
Sector Private 
PI Contribution Demonstrate the potential use of NVIDIA hardware in the development of deep learning embodied models of the numerical cognition in robots.
Collaborator Contribution NVIDIA, as part of the Academic Programs, selected Dr. Di Nuovo's research project and donated to Sheffield Hallam University: (2) NVIDIA Tesla K40 and (1) NVIDIA Titan X graphic cards. The donation aims to empower Dr. Di Nuovo's development of the computational models.
Impact Presentation at the NVIDIA GTC Europe 2018
Start Year 2016
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Manchester
Department School of Computer Science
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Prof. Angelo Cangelosi @ The University of Manchester 
Organisation University of Plymouth
Department Centre for Robotics and Neural Systems (CNRS)
Country United Kingdom 
Sector Academic/University 
PI Contribution PhD Student supervision Visits and discussions about NUMBERS
Collaborator Contribution Visits and discussions about NUMBERS Collaboration in the design of artificial cognitive models
Impact Joint publications and scientific activities, including events organisation, with prof. Cangelosi and other members of his laboratory.
Start Year 2012
 
Description Prof. James McClelland @ Stanford University 
Organisation Stanford University
Department Department of Psychology
Country United States 
Sector Academic/University 
PI Contribution Dr Alessandro Di Nuovo visited the McClelland lab ffrom May to July 2017. During that period, he gave a seminar on the NUMBERS project and interacted with students and other academic members of the lab.
Collaborator Contribution Prof. McClelland is collaborating to the project, providing expert advice in the design and testing of artificial cognitive models, and help with writing an article.
Impact Multi-disciplinary in the intersection of computational intelligence, cognitive science and robotics. A journal article is in preparation.
Start Year 2017
 
Description Article for the Conversation 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact An article for the conversation which has been read by 26265 people (statistics from the website on 01/03/2019), and reprinted by the Independent.
Year(s) Of Engagement Activity 2018
URL https://theconversation.com/robot-carers-could-help-lonely-seniors-theyre-cheering-humans-up-already...
 
Description Invited talk at the NVIDIA European GPU Technology Conference (GTC) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact NVIDIA's GPU Technology Conference (GTC) is a global conference series providing training, insights, and direct access to experts on the hottest topics in computing today.
Year(s) Of Engagement Activity 2018
URL https://tinyurl.com/y974ucy2
 
Description L3C AI in ActIon 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact AI/ML is being applied across multiple industries to provide better insights and enhance decision making. L3C and our sponsors and partners IBM, E2Exchange, ArrowECS have joined together to bring you real examples of where AI/ML is being applied in your industry.

The Royal Institution was founded in 1799 with the aim of introducing new technologies and teaching science. With this heritage in mind we have chosen this esteemed venue for our unique event to show you how AI/ML technology is being used today.

Join us to hear from leaders and innovators showing how AI/ML is being deployed across Insurance, Banking, Retail, Healthcare, Media and HR as well as points of view from leading technology companies such as IBM and Nvidia.

Those with responsibility for ML/AI projects in medium and large enterprises, Heads of ML/AI, BigData, Analytics, Data Scientists as well as those leading or involved in business transformation projects will gain real insight to what is really happening in industries today. Smaller companies looking to disrupt and gain advantage or who address a specific industry issue will gain increased awareness of the state of play.

The day will be a mix of thought provoking and challenging discussions with coffee/networking breaks and lunch culminating in an end of day drinks reception.
Year(s) Of Engagement Activity 2018
URL https://l3c.cloud/l3c-ai-in-action/
 
Description Presentation for the local branch of the British Computer Society (open to public) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Presentation at the local British Computer Society branch, title: Programming robots that care for humans
Date/Time:
Thursday 19 April 2018, 5.30pm - 7.30pm

Venue:
Room 9130, Cantor Building, City Campus, Sheffield Hallam University S1 2ND | Map
Buffet to be served in Room 9020a

The event will start with light refreshments at 5.30pm with the talk commencing at 6.30pm and concluding at 7.30pm

Details:

Recent advances in computing technology and novel artificial intelligence algorithms enable investigation on social applications of robotics, such as the education of children and the care of the elderly. The investigation has the objective to create robots capable of human-like social interaction while exploiting their computational capability for data collection and processing.

Current Social Robots are still very far from those envisioned in science fiction novels, indeed there are only few success cases outside the University labs. Nevertheless, the scientific literature shows the tremendous potential of robotic systems to provide individualised assistance to some classes of people, e.g. children with autism, elderly with dementia.

The talk will present examples from current research projects of Sheffield Robotics at Sheffield Hallam University in the area of socially assistive robotics.

This will show the current state-of-the-art, increase the awareness of benefits and limitation, and discuss the potential breakthroughs and implications for professionals in computing and the general population.

The talk will emphasise responsible user-centred research with the aim to empower people rather than substitute them.
Year(s) Of Engagement Activity 2018
URL https://www.bcs.org/content/ConWebDoc/59182
 
Description Radio Interview for the BBC World program Sounds on How technology can help look after an ageing population 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by the BBC World program the Sounds on How technology can help look after an ageing population.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/sounds/play/w3cswgwk