Echoes 2: Improving Children's Social Interaction through Exploratory Learning in a Multimodal Environment

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description ECHOES II is a technology-enhanced learning (TEL) environment for 5 to 7 years old children to explore and practice skills needed for successful social interaction, such as sharing of attention with others, turn-taking, initiating and responding to bids for interaction. ECHOES supports typically developing children (TD) and children with autism spectrum disorders (ASCs). As well as being a tool for learning by children, ECHOES is a research tool for exploring the specific difficulties of individual children in relation to social interaction.

The project employed participatory design methodology to allow the researchers to co-design the environment with children and teachers and to determine how it could be used in real classrooms. ECHOES also relied on the SCERTS intervention framework and utilised Artificial Intelligence techniques in order to record and interpret interaction data between children and the environment. This data coupled with the data obtained through the ECHOES' evaluation provides the basis for the project's findings and motivates further research.

ECHOES was evaluated in 4 UK schools, over eight weeks. 41 (TD and ASD) children participated. Quantitative analysis of social communication behaviours using a specially designed coding scheme revealed positive trends in improvement in responses to human partner initiations whilst using ECHOES. The improvement observed during ECHOES indicates the potential for TEL environments such as ECHOES to facilitate previously unobserved social communication skills in children with ASC and points the way to further modifications required at the technology engineering level that are needed to engender flexibility and extendability of TEL that fits diverse contexts of use.

The main findings of the project relate to:
(1) efficacy and usefulness of technology enhanced environments in supporting social communication skills by children with ASC. Quantitative analysis of social communication behaviours using a specially designed coding scheme revealed positive trends in improvement in responses to human partner initiations whilst using ECHOES. The improvement observed during ECHOES indicates the potential for TEL environments such as ECHOES to facilitate previously unobserved social communication skills in children with ASC (Bernardini, Porayska-Pomsta, Smith and Avramides, 2012; Bernardini, Porayska-Pomsta and Smith, 2013; Porayska-Pomsta et al., submitted). Some children also showed an improved ability to self-regulate emotionally, especially in the context of the ECHOES task-based activities (Alcorn et al., 2011), to turn-take and share attention with others (Menzies, 2012). Observing such behaviours allowed teachers to appreciate the child's hidden potential and to tailor the support accordingly, thus having instantaneous impact on the specific intervention and practice for the particular children. Observing the individual children to behave more spontaneously and communicatively in the context of ECHOES than in the classroom was reported by the teachers as the key aspect of the ECHOES environment and the reason for their enthusiasm to see technologies such as ECHOES in their classrooms.
(2) importance of technology-enhanced approaches, especially those that utilise Artificial Intelligence methods (user modelling and autonomous agents) both to support learning and research about learning. From the outset, ECHOES was set up to log and interpret the interaction data in real-time. The logging of data is used to capture low-level information (e.g. where the child is touching the screen at any given point; how long does the child takes to respond) and this log data is used to infer patterns of behaviour for each child, to (a) inform the system's design, especially the ability of the virtual character to interact with the child in real-time and (b) inform about each child's progress in minute detail, not feasible in normal classroom circumstances. In attempting to be adaptive based on real-time capture and interpretation of data, coupled with encouraging results and enthusiastic response by teachers, ECHOES provides an early and important step towards adaptive technologies for autism. The logged data also provides a precious resource for the generation of further hypotheses related to the individual children and children with ASC more generally, of relevance to psychologists, education experts and computer scientists (Bernardini, Porayska-Pomsta and Smith, 2013).
(3) participatory design (PD) methods involving young children with ASD and their carers in the design of the technologies for use by them. ECHOES invested in PD from the start and has successfully extended and tested the existing methods of involving young children with autism and their teachers in the design of technologies for use by them (Frauenberger, Good, and Keay-Bright, 2011), thereby contributing further to human-computer interaction research.
Exploitation Route The findings of the ECHOES II project have already been taken forward in a multitude of different follow-on projects, including the ESRC funded Shape project (ES/J011207/1) which aimed to explore the way ECHOES, amongst other three technology-enhanced learning environments for autism, fared in the context of real classrooms with minimal support from the researchers. The key to the Shape project was the need to move beyond the aspirational rhetoric of TEL for real classroom to investigating the pre-requisites of real-world uptake.

The EPSRC funded SHARE-IT project (EP/K012428/1) also derived as a direct consequence of ECHOES II. One of the key practical findings of ECHOES related to the need to develop industry standard software as a means of fulfilling the ambition of TEL making it to the real-world. Another finding related to the need for the technology to be flexible and adaptable to the specific contexts and circumstances in which learning experiences are offered. Several children during ECHOES evaluation studies manifested behaviours which they did not in the class, highlighting the need for between-contexts intervention and support. Therefore, SHARE-IT's main efforts were dedicated to building, in partnership with parents, teachers and children, a robust technology, which could be also authored by the users (without substantial involvement of researchers) to suit the contexts in which they wanted to use the environment.

The SHARE-IT project was fundamental to securing further funding in what is becoming a larger research programme related to TEL design, implementation and uptake. Specifically the TESSA project, a small, but important Institute of Education funded knowledge transfer project (£26K), relates to exploring inter-cultural viability of the SHARE-IT environment (and through it of many of the ECHOES original ideas) to other cultural contexts - in this case to the context of Indian education. The unLOCKE project, funded jointly by the Wellcome Trust and Educational Endowment Foundation (£1M) reflects a step-change in both the quality and general applicability of original ideas and findings of ECHOES and of extendability of the technology developed through SHARE-IT (its child project). unLOCKE aims to create a neuroscientific intervention for training primary school children's inhibition of intuitive reasoning in relation to counterintuitive concepts in mathematics and science (http://educationendowmentfoundation.org.uk/projects/counterintuitive-concepts/).
Sectors Digital/Communication/Information Technologies (including Software),Education,Healthcare
URL http://www.echoes2.org