Echoes 2: Improving Children's Social Interaction through Exploratory Learning in a Multimodal Environment

Lead Research Organisation: Institute of Education
Department Name: Culture, Communication and Media

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.

Publications

10 25 50
 
Title Emerging technologies : echoes 
Description Produced by Xube and commissioned by Becta this video showcases the TLRP-TEL Echoes project led by Kaska Porayska-Pomsta at the London Knowledge Lab. The project is looking at the development of social skills in Asperger's Spectrum and typically developing 5-7 year olds using multi-touch screens, AI and gaze and motion capture technology among other technologies in a multimodal environment. 
Type Of Art Film/Video/Animation 
Year Produced 2010 
 
Description ECHOES II is a technology-enhanced learning (TEL) environment for 5 to 7 years old children to explore and practice skills needed for successful social interaction, such as sharing of attention with others, turn-taking, initiating and responding to bids for interaction. ECHOES supports typically developing children (TD) and children with autism spectrum disorders (ASCs). As well as being a tool for learning by children, ECHOES is a research tool for exploring the specific difficulties of individual children in relation to social interaction.

The project employed participatory design methodology to allow the researchers to co-design the environment with children and teachers and to determine how it could be used in real classrooms. ECHOES also relied on the SCERTS intervention framework and utilised Artificial Intelligence techniques in order to record and interpret interaction data between children and the environment. This data coupled with the data obtained through the ECHOES' evaluation provides the basis for the project's findings and motivates further research.

ECHOES was evaluated in 4 UK schools, over eight weeks. 41 (TD and ASD) children participated. Quantitative analysis of social communication behaviours using a specially designed coding scheme revealed positive trends in improvement in responses to human partner initiations whilst using ECHOES. The improvement observed during ECHOES indicates the potential for TEL environments such as ECHOES to facilitate previously unobserved social communication skills in children with ASC and points the way to further modifications required at the technology engineering level that are needed to engender flexibility and extendability of TEL that fits diverse contexts of use.

The main findings of the project relate to:
(1) efficacy and usefulness of technology enhanced environments in supporting social communication skills by children with ASC. Quantitative analysis of social communication behaviours using a specially designed coding scheme revealed positive trends in improvement in responses to human partner initiations whilst using ECHOES. The improvement observed during ECHOES indicates the potential for TEL environments such as ECHOES to facilitate previously unobserved social communication skills in children with ASC (Bernardini, Porayska-Pomsta, Smith and Avramides, 2012; Bernardini, Porayska-Pomsta and Smith, 2013; Porayska-Pomsta et al., submitted). Some children also showed an improved ability to self-regulate emotionally, especially in the context of the ECHOES task-based activities (Alcorn et al., 2011), to turn-take and share attention with others (Menzies, 2012). Observing such behaviours allowed teachers to appreciate the child's hidden potential and to tailor the support accordingly, thus having instantaneous impact on the specific intervention and practice for the particular children. Observing the individual children to behave more spontaneously and communicatively in the context of ECHOES than in the classroom was reported by the teachers as the key aspect of the ECHOES environment and the reason for their enthusiasm to see technologies such as ECHOES in their classrooms.
(2) importance of technology-enhanced approaches, especially those that utilise Artificial Intelligence methods (user modelling and autonomous agents) both to support learning and research about learning. From the outset, ECHOES was set up to log and interpret the interaction data in real-time. The logging of data is used to capture low-level information (e.g. where the child is touching the screen at any given point; how long does the child takes to respond) and this log data is used to infer patterns of behaviour for each child, to (a) inform the system's design, especially the ability of the virtual character to interact with the child in real-time and (b) inform about each child's progress in minute detail, not feasible in normal classroom circumstances. In attempting to be adaptive based on real-time capture and interpretation of data, coupled with encouraging results and enthusiastic response by teachers, ECHOES provides an early and important step towards adaptive technologies for autism. The logged data also provides a precious resource for the generation of further hypotheses related to the individual children and children with ASC more generally, of relevance to psychologists, education experts and computer scientists (Bernardini, Porayska-Pomsta and Smith, 2013).
(3) participatory design (PD) methods involving young children with ASD and their carers in the design of the technologies for use by them. ECHOES invested in PD from the start and has successfully extended and tested the existing methods of involving young children with autism and their teachers in the design of technologies for use by them (Frauenberger, Good, and Keay-Bright, 2011), thereby contributing further to human-computer interaction research.
Exploitation Route The findings of the ECHOES II project have already been taken forward in a multitude of different follow-on projects, including the ESRC funded Shape project (ES/J011207/1) which aimed to explore the way ECHOES, amongst other three technology-enhanced learning environments for autism, fared in the context of real classrooms with minimal support from the researchers. The key to the Shape project was the need to move beyond the aspirational rhetoric of TEL for real classroom to investigating the pre-requisites of real-world uptake.

The EPSRC funded SHARE-IT project (EP/K012428/1) also derived as a direct consequence of ECHOES II. One of the key practical findings of ECHOES related to the need to develop industry standard software as a means of fulfilling the ambition of TEL making it to the real-world. Another finding related to the need for the technology to be flexible and adaptable to the specific contexts and circumstances in which learning experiences are offered. Several children during ECHOES evaluation studies manifested behaviours which they did not in the class, highlighting the need for between-contexts intervention and support. Therefore, SHARE-IT's main efforts were dedicated to building, in partnership with parents, teachers and children, a robust technology, which could be also authored by the users (without substantial involvement of researchers) to suit the contexts in which they wanted to use the environment.

The SHARE-IT project was fundamental to securing further funding in what is becoming a larger research programme related to TEL design, implementation and uptake. Specifically the TESSA project, a small, but important Institute of Education funded knowledge transfer project (£26K), relates to exploring inter-cultural viability of the SHARE-IT environment (and through it of many of the ECHOES original ideas) to other cultural contexts - in this case to the context of Indian education. The unLOCKE project, funded jointly by the Wellcome Trust and Educational Endowment Foundation (£1M) reflects a step-change in both the quality and general applicability of original ideas and findings of ECHOES and of extendability of the technology developed through SHARE-IT (its child project). unLOCKE aims to create a neuroscientific intervention for training primary school children's inhibition of intuitive reasoning in relation to counterintuitive concepts in mathematics and science (http://educationendowmentfoundation.org.uk/projects/counterintuitive-concepts/).
Sectors Digital/Communication/Information Technologies (including Software),Education,Healthcare

URL http://echoes2.org/
 
Description The ECHOES project has generated much interest both within the scientific community as well as in non-academic communities, primarily amongst teachers and parents. This is reflected in the number of publications, both journal and conference, which have been produced to date, with a number of journal publications still being either under review or have been just accepted for publication. The project attracts multi-disciplinary interest with both social scientists and computer scientists being equally keen to understand and reuse the ECHOES technologies and approaches. We have had interest from industrial companies, two of which have been actively involved in the follow-on SHARE-IT project (EP/K012428/1) and have donated an equivalent of £50K in equipment and labour to date. Another important impact outcome is a lasting and growing collaboration with schools and their enthusiasm for the ECHOES technology and the research process. This is evidenced in their continuing involvement in the follow-on projects, including the Shape (ES/J011207/1) and the SHARE-IT projects. International collaborations are flourishing as a direct consequence of the project and its success. The PI of ECHOES has been invited to participate in a EU-FP7 project (called TARDIS) which builds on methods for both designing and evaluating intelligent learning environment for supporting the development of social communication. International students are increasingly attracted to the partner institutions. One of the notable collaborations is the one between the ECHOES PI and IIIT-Hyderbad, which has led directly to the EPSRC funded SHARE-IT project and TESSA (Institute of Education funded knowledge transfer project) with further funded collaborations being planned in the foresable future. Two high profile projects have been funded independently by the ESRC and EPSRC since the end of the ECHOES project to the combined value of £370K (fEC), demonstrating that both councils and the reviewers of the proposals regard the ECHOES project as a solid basis for further social scientific and computer science research. Further funding has been granted to utilise the SHARE-IT platform, which has been inspired substantially by ECHOES, in delivering neuroscientific intervention in ninety schools in the UK (the unLOCKE project, funded jointly by the Wellcome Trust and the Educational Endowment Foundation). We now have four schools keenly involved in the follow-on research and a commitment from the teachers and headteachers to continue this involvement indefinitely. The schools are investing their own money and time to be involved in the research led by us and are putting in place practices which reflect the lessons learned during the ECHOES project as well as advanced technologies including ECHOES and its new incarnation developed through SHARE-IT. The academic impact has been achieved chiefly through publishing the results of the ECHOES research as well as through invited talks and high profile events such as the Royal Society Event and the Festival of Social Science. Both events attracted media attention, including Sky News, BBC online, Polish National Television, Deutsche Welle Radio as well as ESRC inhouse publications and Press Releases, including the Impact video about ECHOES. Our continuing collaboration and dialogue with schools is the chief reason for the non-academic impact. We believe that the continuity of engagement is key to developing trust between teachers, parents and children and we have continued our work with all stakeholders even when funding was not available. Researchers who worked on the ECHOES project have benefited a great deal in terms of the interdisciplinary methods and practices that they learned throughout the project. This knowledge is now imparted on the new generation of TEL researchers at PhD and masters levels at all partner institutions, with many a master project focusing on ECHOES - either its possible redesigns, further participatory design methods applications and data analysis. Internships and research visits by international researchers are common and they lead to further dissemination of the project as well as new collaborations, new perspectives and new opportunities for impact. Industrial companies are now involved in furthering the real-world robustness of the technology prototypes during ECHOES. Many teachers have changed their attitude towards technology and their understanding of what technology is and can do. The main impact in this respect was a shift in a number of teachers' interpretations of their role in the design process and their realisation that instead of being merely consumers of technologies designed by others, they can be co-creators and curators of technology. This shift in attitude has lead directly to the definition of the objectives for both of the follow-on projects funded.
First Year Of Impact 2012
Sector Digital/Communication/Information Technologies (including Software),Education,Healthcare
Impact Types Societal

 
Description School-Home Research Environment through Intelligent Technologies
Amount £301,999 (GBP)
Funding ID EP/K012428/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2013 
End 07/2014
 
Description Shaping the future of technology use in the classroom
Amount £69,684 (GBP)
Funding ID ES/J011207/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 05/2012 
End 06/2013
 
Description TESSA: UK-India intercultural knowledge transfer in technology-enhanced school and home support for autism spectrum conditions.
Amount £26,000 (GBP)
Organisation University College London 
Department Institute of Education (IOE)
Sector Academic/University
Country United Kingdom
Start 08/2014 
End 07/2015
 
Description unLOCKE: Learning Counterintuitive Concepts
Amount £1,000,000 (GBP)
Organisation Wings for Life Spinal Cord Research Foundation 
Sector Private
Country Austria
Start 01/2015 
End 12/2019
 
Description Festival of Social Sciences November 2012 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact We organised an ESRC Festival of Social Sciences in relation to the Shape project. The event took place at Topcliffe Primary School in Birmingham where installations of the four technologies investigated by the project, including the ECHOES learning environment.

The event received much attention from the public, including parents and teachers from schools across the country and much media attention, iincluding Sky News, BBC online, Polish National TV and a large following on social networking sites.
Year(s) Of Engagement Activity 2012
 
Description Public debate 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The debate attracted nearly 100 audience participants who engaged with the central question related to the future of technology for autistic people and the ethical issues surrounding its development and use.
Year(s) Of Engagement Activity 2016
URL http://crae.ioe.ac.uk/post/137146874478/special-crae-discussion-panel-autism-and
 
Description Serious games as intervention tools for the socially excluded 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Invited lecture at the Serious Games Institute in Coventry. The talk led to important discussion related to the use of technology for social inclusion as well as the use of participatory design methods as a way of informing the implementation of technology and its use in real-world contexts.

The talk has led the extension of the collaboration with other academics.
Year(s) Of Engagement Activity 2013