Cross-modal Interactive Tools for Inclusive Learning
Lead Research Organisation:
University of Bristol
Department Name: Computer Science
Abstract
Students with special education needs are traditionally placed in specialized schools where they are provided with facilities and trained staff to accommodate their needs. Today, there is a shift in government policies towards providing support for equalisation of education opportunity, embodying UNESCO's Salamanca World Statement which considers "Inclusion and participation [as] essential to human dignity [...] enjoyment and exercise of human rights" (UNESCO 1994). This shift underlines the national agenda for integrating more visually impaired children in mainstream schools and reducing exclusion, which is often the result of unmet special needs.
According to a recent report by the Royal National Institute for the Blind (RNIB), approximately 70 per cent of visually impaired children in the UK are educated in mainstream settings. This often takes the form of one or two learners in a class of fully sighted peers. Issues related to how best to modify learning materials as well as how to manage group work to nurture an adequate learning environment for all is challenging in such settings. For instance, a number of curriculum standards of core components in science, technology, engineering and mathematics (STEM) emphasise the importance of graphic literacy to establish a foundation for future practices. Teachers in mainstream schools therefore have to resort to modifying curriculum resources that include graphics to make them accessible to visually impaired learners by using physical tools such as Braille, tactile diagrams or heat-raised images. However, these tools are designed to be used by visually impaired learners alone and not by their sighted peers, and thus can end up forcing them to learn as isolated individuals and exclude from group learning activities. In addition, as classrooms become more computerised, visually impaired learners are likely to face with more barriers since they mostly rely on screen-readers to access computers and these can be inadequate for accessing graphics and can inhibit meaningful collaboration with sighted peers. If not carefully studied and designed, technology can have detrimental effects on the inclusion of visually impaired learners in mainstream schools, particularly in contexts where they interact with sighted peers, and can lead to drastic effects on their lives as adults. For instance, jobs that require data analysis, including careers in STEM, often encode information using graphics and require teamwork and collaboration. Learning how to interpret and construct graphics as well as how to effectively engage with peers should, and generally does, begin during a student's basic education.
The underlying problem is that current technological support for learning in mixed classrooms emphasise accessibility over inclusion, targeting individual rather than social learning and group work. Focus is thus placed on an individual's disability and not on the variety of abilities present in a social context of group learning involving students, teachers and technology.
The aim of this Early Career fellowship is to research and develop interactive learning tools to make mixed classrooms more inclusive of visually impaired students. Non-visual modalities (e.g. audio, gestures, haptic and tactile feedback) have already shown potential benefit to support accessible interactions, but there are still limitations in their applicability in real world settings, such as issues with cross-modal effects when groups work together using different senses. The developed tools will focus on gaps in technological support for accommodating national curriculum standards by combining participatory design activities with empirical research into cross-modal interaction to find out how different senses can be integrated with visual capabilities. The tools will be validated in classroom settings to find out how they can improve group learning activities and teaching practices.
According to a recent report by the Royal National Institute for the Blind (RNIB), approximately 70 per cent of visually impaired children in the UK are educated in mainstream settings. This often takes the form of one or two learners in a class of fully sighted peers. Issues related to how best to modify learning materials as well as how to manage group work to nurture an adequate learning environment for all is challenging in such settings. For instance, a number of curriculum standards of core components in science, technology, engineering and mathematics (STEM) emphasise the importance of graphic literacy to establish a foundation for future practices. Teachers in mainstream schools therefore have to resort to modifying curriculum resources that include graphics to make them accessible to visually impaired learners by using physical tools such as Braille, tactile diagrams or heat-raised images. However, these tools are designed to be used by visually impaired learners alone and not by their sighted peers, and thus can end up forcing them to learn as isolated individuals and exclude from group learning activities. In addition, as classrooms become more computerised, visually impaired learners are likely to face with more barriers since they mostly rely on screen-readers to access computers and these can be inadequate for accessing graphics and can inhibit meaningful collaboration with sighted peers. If not carefully studied and designed, technology can have detrimental effects on the inclusion of visually impaired learners in mainstream schools, particularly in contexts where they interact with sighted peers, and can lead to drastic effects on their lives as adults. For instance, jobs that require data analysis, including careers in STEM, often encode information using graphics and require teamwork and collaboration. Learning how to interpret and construct graphics as well as how to effectively engage with peers should, and generally does, begin during a student's basic education.
The underlying problem is that current technological support for learning in mixed classrooms emphasise accessibility over inclusion, targeting individual rather than social learning and group work. Focus is thus placed on an individual's disability and not on the variety of abilities present in a social context of group learning involving students, teachers and technology.
The aim of this Early Career fellowship is to research and develop interactive learning tools to make mixed classrooms more inclusive of visually impaired students. Non-visual modalities (e.g. audio, gestures, haptic and tactile feedback) have already shown potential benefit to support accessible interactions, but there are still limitations in their applicability in real world settings, such as issues with cross-modal effects when groups work together using different senses. The developed tools will focus on gaps in technological support for accommodating national curriculum standards by combining participatory design activities with empirical research into cross-modal interaction to find out how different senses can be integrated with visual capabilities. The tools will be validated in classroom settings to find out how they can improve group learning activities and teaching practices.
Planned Impact
The outcomes of the project will increase the inclusion of students with visual impairments in mixed classrooms. Students and teachers taking part in the various design activities and studies during the course of the project will feel immediate impact through exposure to new technologies for inclusive collaboration. I will achieve this by involving the Sensory Support Services and the King's Copse Primary School when recruiting participants, as well as existing links with other user groups and bodies such as the RNIB to disseminate the work to the educational sector. Findings from this research will provide practical examples of how it will impact teaching practices and group learning activities. In the near term this impact will be felt by increased inclusion in mainstream schools, and in the medium term findings from this project will impact teams working with collaborative tools in a range of domains. The user-centred approach used in this fellowship will help raise awareness of the conducted research through inclusion of target user groups in research and design activities. I will also raise awareness through publicity in national and international specialist and mainstream media.
In the long term the project will have social and economic benefit to the UK, as it will allow a wider range of people to engage in collaborative work. It will also benefit the general populace with the design of more effective cross-modal collaboration support for situations when modalities are restricted (e.g. distributed and mobile teamwork). Additionally, it will contribute to making the UK an example of best practice for inclusion in education and teamwork. The UK commercial sector will benefit from this research through the advancement of knowledge in collaborative technologies. Moreover, the software developed in this project will help improve the UK's position in the assistive technology market by opening up the collaboration domain. Furthermore, by increasing inclusion in collaborative work, there will be an increase in the efficiency of teamwork, and increased dissemination and sharing of knowledge. In order to achieve this, the deliverable software tools will be made available to UK companies and organizations free of charge through an open source license. Furthermore, experience in building cross-modal systems will be made publicly available through the open source community and academic papers which will provide the commercial and educational sectors with case studies of how to undertake such development. I will raise awareness of the developed software tools through public exhibitions and media channels.
In the long term the project will have social and economic benefit to the UK, as it will allow a wider range of people to engage in collaborative work. It will also benefit the general populace with the design of more effective cross-modal collaboration support for situations when modalities are restricted (e.g. distributed and mobile teamwork). Additionally, it will contribute to making the UK an example of best practice for inclusion in education and teamwork. The UK commercial sector will benefit from this research through the advancement of knowledge in collaborative technologies. Moreover, the software developed in this project will help improve the UK's position in the assistive technology market by opening up the collaboration domain. Furthermore, by increasing inclusion in collaborative work, there will be an increase in the efficiency of teamwork, and increased dissemination and sharing of knowledge. In order to achieve this, the deliverable software tools will be made available to UK companies and organizations free of charge through an open source license. Furthermore, experience in building cross-modal systems will be made publicly available through the open source community and academic papers which will provide the commercial and educational sectors with case studies of how to undertake such development. I will raise awareness of the developed software tools through public exhibitions and media channels.
Organisations
- University of Bristol (Fellow, Lead Research Organisation)
- Sensory Support Service (Collaboration)
- Toulouse Institute of Computer Science Research (Collaboration)
- Southville Primary School (Collaboration)
- Emersons Green Primary School (Collaboration)
- Microsoft Research (Collaboration)
- Special Educational Needs Support Services (Collaboration)
- Oxford Spires Academy (Collaboration)
- The Emmbrook School (Collaboration)
- Hamad bin Khalifa University (Collaboration)
- Berkshire Sensory Consortium Service (Collaboration)
- John Hampden Primary School (Collaboration)
- University of Bristol (Collaboration)
People |
ORCID iD |
| Oussama Metatla (Principal Investigator / Fellow) |
Publications
Metatla O
(2016)
Tap the ShapeTones
Cullen C
(2018)
Multisensory storytelling
Metatla O
(2018)
Inclusive Education Technologies
Krüger M
(2019)
What is participation? emerging challenges for participatory design in globalized conditions
in Interactions
Emeline Brule
(2019)
Evaluating Technologies with and for Disabled Children.
Metatla O
(2019)
Voice User Interfaces in Schools
Oussama Metatla
(2019)
Toward classroom experiences inclusive of students with disabilities
in ACM Interactions
Metatla O
(2019)
"Like Popcorn"
Cullen C
(2020)
Tangible Multisensory Aids for Collaborative Phonics Learning
Metatla O
(2020)
Enabling children to design for others with expanded proxy design
| Description | Findings from ethnographic and qualitative studies - involving local authorities support services and a number of mainstream schools that include pupils with visual impairments - are helping to characterise the various conceptual and practical aspects of inclusion as viewed by support practitioners, teachers and pupils; i.e. providing answers to the question: what are the social, organisational, pedagogical, environmental and interactional elements that contribute to a rounded picture of what constitutes inclusion of visually-impaired pupils in mainstream schools. These key findings are contributing to meeting the first main objective of this project. The results of the ethnographic studies have now yielded a full characterisation of inclusion of visually impaired children in mainstream schools in terms of six main themes: 1) Coordination and planning of support teams, 2) Learning experience, 3) Social engagement, 4) Mobility, 5) Artefacts, materials and tools, and 6) Extended curriculum. We have also identified the "assistance bubble" as an overarching theme that characterise the learning experience of children living with visual impairments in mainstream schools, and we showed through this construct, how existing educational and social support structures could lead to exclusion from group learning, social play and independent mobility. To date, we have also demonstrated how adaptations of co-designing methods for children with mixed visual abilities open up novel design spaces and produce conceptual designs for technology that can accommodate and augment the varied abilities present in inclusive schooling environments. This contributes to meeting the second objective of the project: i.e. devising participatory design methods for designing novel mutlisensory tools to support group learning activities, including strategies for involving teachers and both visually and sighted peers in conceptual design. Most importantly, we are articulating a framework for inclusive co-design with mixed-ability groups that support children with and without VIs to jointly lead design processes and to effectively conceptualise designs that reflect considerations for mixed visual abilities with potentials to bridge social disconnects in their shared learning environments. We have also explored how crossmodal correspondences paradigms could be exploited in the design of inclusive education technologies. We conducted the first exploration of the "Bouba/Kiki" crossmodal correspondence effect in a concurrent presentation of tangible and olfactory form. We examined the effect with children in crossmodal and emotional association tasks and found some evidence that supports pre-existing mappings, and confirmed the presence of crossmodal associations between angular shapes and the lemon scent with arousing emotion, and round shapes and the vanilla scent with calm- ing emotion. We have used these findings to inform the design and implementation of novel multisensory education technologies in areas ranging from storytelling to peer-revision and mobility. We find that in the context of inclusive interaction with mixed ability groups, principles of crossmodal interaction are a promising line of enquiry for inclusive activities, particularly for collaborative work with children with special educational needs and disabilities. For example, we have used these findings to guide the implementation of collaborative storytelling tools, and explored how voice-based interfaces, e.g. Amazon Echo, could be augmented with multisensory interaction and used outside home settings and inside schools. Outcomes include a design space for thinking about inclusive education through VUIs, a set of scenarios that illustrate possible uses of VUIs in inclusive education, and an example prototype application validating one of the identified scenarios and helping extend the design space to the realm of multisensory, crossmodal and tangible interaction. More recently, we used a co-design approach to design and evaluate a robot-based educational game that could be inclusive of both visually impaired and sighted children in the context of mainstream education. We ran a focus group discussion with visual impairment educators to understand barriers to inclusive social play. And then a series of co-design workshops to engage visually impaired and sighted children and educators in learning about and critiquing a commodity robot technology and exploring its potential to support inclusive play experiences. We presented design guidelines and an evaluation workshop of a game prototype, demonstrating group dynamics conducive to collaborative learning experiences, including shared goal setting/execution, closely coupled division of labour, and interaction symmetry. We also examined further corssmodal correspondences between tangible haptic interaction colour and emotions. We contribute novel knowledge of how tangible, visual and emotional features are perceived and interpreted alongside one another, which helps map out untapped space of crossmodal correspondences (CCs) in HCI. We make 4 key contributions to this space in terms of 1) more complex stimuli 2) active touch 3) transitional features and 4) measurement: 1) Studies that assessed CCs/touch looked at 2D shapes, word association, vibration, single-point haptic contact, and scent-touch stimuli. We contribute evidence of CCs for more complex tangible stimuli, extending single point protrusion stimuli. We also extend prior work by placing CCs in the more naturalistic context of active touch exploration/. Importantly 1) and 2) bring CCs closer to HCI design space. We also show new evidence for transitional features of CCs between tangible, visual and emotional dimensions. Specifically, tangible and visual features of angularity and brightness share emotional valence. Here, our findings highlight limitations to the linear predictability of CC translation which is an important consideration for designers of multisensory HCI. Most recently, we examined crossmodal correspondences in shape-changing displays. With the proliferation of shape-change research in affective computing, there is a need to deepen understandings of affective responses to shape-change display. Little research has focused on affective reactions to tactile experiences in shape-change, particularly in the absence of visual information. It is also rare to study response to the shape-change as it unfolds, isolated from a final shape-change outcome. We ran two studies on touch-affect associations, using the crossmodal "Bouba-Kiki" paradigm, to understand affective responses to shape-change as it unfolds. We investigated experiences with a shape-change gadget, as it moves between rounded ("Bouba") and spiky ("Kiki") forms. We capture affective responses via the circumplex model, and use a motion analysis approach to understand the certainty of these responses. We found that touch-affect associations are influenced by both the size and the frequency of the shape-change and may be modality-dependent, and that certainty in affective associations is influenced by association-consistency. |
| Exploitation Route | The CRITICAL research agenda is to explore and demonstrate how HCI, as an applied field of inquiry, can contribute to making human society more inclusive of people with disabilities. At the core of this agenda are fundamental questions around what an inclusive technology is, and a need to re-examine approaches to designing accessible and assistive technologies. Often, this has traditionally focused on addressing functional accessibility issues by foregrounding the needs of a single user group. CRITICAL research has demonstrated that this can work against objectives of inclusion, leading to solutions that place more emphasis on what a disabled individual can and cannot do rather than on the interplay between social and technological structures situating their actions, which is important to both characterise and address barriers to inclusion. Findings from the CRITICAL research have helped us in articulating and pursuing an alternative line of inquiry in order to understand whether and how inclusive technologies should be demarcated from traditional assistive technology through design, development and evaluation. The above findings can be used to drive the design and development of technological and practical support for inclusion of pupils with visual impairments in mainstream schools. In the context of this project, these initial findings are currently being used to focus further follow-up co-design activities involving end users (support services practitioners, teachers and pupils) to create conceptual designs for a variety of multisensory interactive technologies. These designs will thus embody end users' views and perspectives on inclusion and how it can be improved in mainstream schools. The adaptation of co-design techniques contribute to expanding the repertoire of methods and techniques that could be used to engage children in design: We demonstrated how methods known to be effective in early conceptual design but not previously used to engage participants with mixed-visual abilities (e.g bodystorming) can be adapted to this new context. Thus these techniques shape the potential to contribute to new, more inclusive design practices in this space. Our inclusive co-design framework thus extends work on interaction design with children and on multisensory and crossmodal interaction. Work on crossmodal correspondences extend prior work in this area by increasing our understanding of how sensory modalities interact with and relate to one another. They also provide novel insights for informing the design of richer and more engaging multisensory experiences. For example, work on voice-based user interfaces, robots, and shape-changing displays provide a basis for designing support for inclusive education through these interactive technologies and contribute to extending the design space of potential applications across a number of settings, thus providing a foundation for future design and research in these area. The above findings can be used to drive the design and development of technological and practical support for inclusion of pupils with visual impairments in mainstream schools. In the context of this project, these initial findings are currently being used to focus further follow-up co-design activities involving end users (support services practitioners, teachers and pupils) to create conceptual designs for a variety of multisensory interactive technologies. These designs will thus embody end users' views and perspectives on inclusion and how it can be improved in mainstream schools. The adaptation of co-design techniques contribute to expanding the repertoire of methods and techniques that could be used to engage children in design: We demonstrated how methods known to be effective in early conceptual design but not previously used to engage participants with mixed-visual abilities (e.g bodystorming) can be adapted to this new context. Thus these techniques shape the potential to contribute to new, more inclusive design practices in this space. Our inclusive co-design framework thus extends work on interaction design with children and on multisensory and crossmodal interaction. Work on crossmodal correspondences extend prior work in this area by increasing our understanding of how sensory modalities interact with and relate to one another. They also provide novel insights for informing the design of richer and more engaging multisensory experiences. For example, work on voice-based user interfaces and robots provide a basis for designing support for inclusive education through these interactive technologies and contribute to extending the design space of potential applications across a number of settings, thus providing a foundation for future design and research in these area. |
| Sectors | Communities and Social Services/Policy,Creative Economy,Digital/Communication/Information Technologies (including Software),Education |
| Description | Work carried out as part of the CRITICAL project has been presented at non-academic venues to engage the general public and specialist end user groups, including educators and children with and without visual impairments as well as practitioners who specialise in providing support to children with visual impairments and their families. Events include presentations at the VIEW conference, an annual conference for educators and practitioners in the areas of education and development of children and young people with vision impairment; demonstrations at We The Curious; a half day visit and workshop event at the Bristol museum, participatory design workshops at a number of primary and secondary schools in the areas of Bristol and Oxford, and an inclusive hackathon event. As part of CRITICAL, we have also been organising a series of workshops, panels and special interest group meetings (e.g. at CHI2018, CHI2019, ICMI2020 and IDC2020, CHI2021, IDC2022), as part of efforts of creating a consortium and community of researchers and practitioners to define and shape research and practice directions in the areas of inclusive technologies, particular in terms of how inclusive technologies, as an approach for thinking about technologies that can be used by both disabled and non-disabled people, should be demarcated from assistive technologies in terms of design, development and evaluation. |
| First Year Of Impact | 2020 |
| Sector | Communities and Social Services/Policy,Education,Culture, Heritage, Museums and Collections |
| Impact Types | Cultural,Societal,Policy & public services |
| Description | All-Party Parliamentary Group for Assistive Technology |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Qualified Teacher for Visual Impairment Practice Training |
| Geographic Reach | Local/Municipal/Regional |
| Policy Influence Type | Influenced training of practitioners or researchers |
| Impact | Involved trainee practitioner (training to become qualified teachers for visual impairments) in observing and engaging with participatory design activities designed to research and develop novel educational technologies for visual impairment. They have used their involvement in the project as part of their practice assignments for obtaining said qualifications. We have also influenced their views on inclusive and accessible technologies, which should have an impact on their outlook on educational technologies and pedagogy during their practice. |
| Description | Brigstow Institute Seedcorn Funding |
| Amount | £7,529 (GBP) |
| Organisation | University of Bristol |
| Department | Brigstow Institute |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 02/2018 |
| End | 07/2018 |
| Description | Inclusive Cross-sensory Social Play: Towards a new paradigm of assistive technology for early development of blind and visually impaired children |
| Amount | € 1,970,000 (EUR) |
| Organisation | European Research Council (ERC) |
| Sector | Public |
| Country | Belgium |
| Start | 09/2023 |
| End | 08/2028 |
| Description | QNRF - NATIONAL PRIORITY RESEARCH PROGRAM |
| Amount | £170,000 (GBP) |
| Organisation | Qatar Foundation |
| Department | Qatar National Research Fund |
| Sector | Charity/Non Profit |
| Country | Qatar |
| Start | 09/2021 |
| End | 08/2024 |
| Description | Telling Tales of Engagement |
| Amount | £10,000 (GBP) |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 01/2021 |
| End | 09/2021 |
| Description | Using AI to develop joint attention in blind children |
| Amount | £119,928 (GBP) |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2020 |
| End | 03/2024 |
| Description | Department of Education, University of Bristol |
| Organisation | University of Bristol |
| Department | School of Education |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We are collaborating with researchers from the School of Education to develop design and evaluation techniques grounded in pedagogical and educational principles |
| Collaborator Contribution | The researchers from the School of Education bring expertise in pedagogical and educational principles, and which contribute directly to grounding our investigations of inclusive education technologies for visually impaired and sighted children in mainstream schools |
| Impact | None yet |
| Start Year | 2018 |
| Description | Emersons Green Primary School |
| Organisation | Emersons Green Primary School |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We have visited the school, coordinated with the school's Special Education Needs Coordinator to organise discussions with staff and pupils regarding their potential involvement in the development of accessibility technology to support visually impaired children during group work |
| Collaborator Contribution | The school has provided us with access to their staff and pupils who are currently taking part in an on going ethnographic study involving observations of teaching sessions, and interviews with teachers, staff and pupils. Future activities will also involve design workshop and technology evaluation sessions |
| Impact | We were able to run an ethnographic study at this school (on going), which is helping us develop a rounded picture of the inclusion of visually impaired children in mainstream schools. This feeds directly into the process of designing technology to support inclusive learning and teaching practices |
| Start Year | 2016 |
| Description | IRIT University of Toulouse |
| Organisation | Toulouse Institute of Computer Science Research |
| Country | France |
| Sector | Private |
| PI Contribution | Designed and developed part of the experimental apparatus for a study of robot-based interaction. Designed and ran a series of co-design workshops and evaluation study with visually-impaired and sighted children on social play in mainstream schools. Co-wrote papers reporting on results |
| Collaborator Contribution | Designed and developed part of the social play inclusive game used in the study. Ran a workshop with game designers to trial out design insights generated from previous workshops. Co-wrote papers reporting on results |
| Impact | ACM SIGCHI Paper (Best Paper Award) |
| Start Year | 2019 |
| Description | John Hampden Primary School |
| Organisation | John Hampden Primary School |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | We are currently in contact with this school, coordinating with the school's Special Education Needs Coordinator to organise discussions with staff and pupils regarding their potential involvement in the development of accessibility technology to support visually impaired children during group work |
| Collaborator Contribution | The school will provide us with access to their staff and pupils who will take part in an on going ethnographic study involving observations of teaching sessions, and interviews with teachers, staff and pupils. Future activities will also involve design workshop and technology evaluation sessions |
| Impact | none yet |
| Start Year | 2017 |
| Description | Oxford Spires Academy |
| Organisation | Oxford Spires Academy |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We have visited the school, coordinated with the school's Special Education Needs Coordinator to organise discussions with staff and pupils regarding their potential involvement in the development of accessibility technology to support visually impaired children during group work |
| Collaborator Contribution | The school has provided us with access to their staff and pupils who are currently taking part in an on going ethnographic study involving observations of teaching sessions, and interviews with teachers, staff and pupils. Future activities will also involve design workshop and technology evaluation sessions |
| Impact | We were able to run an ethnographic study at this school (on going), which is helping us develop a rounded picture of the inclusion of visually impaired children in mainstream schools. This feeds directly into the process of designing technology to support inclusive learning and teaching practices |
| Start Year | 2017 |
| Description | Qatar Foundation HBKU partnership |
| Organisation | Hamad bin Khalifa University |
| Country | Qatar |
| Sector | Academic/University |
| PI Contribution | Joint Qatar Foundation award |
| Collaborator Contribution | Joint Qatar Foundation award |
| Impact | Non yet |
| Start Year | 2021 |
| Description | SEN Support Services Oxford |
| Organisation | Special Educational Needs Support Services |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | The Special Educational Need Support Service, is part of Oxford City Council, is a service providing support for children with sensory impairments in mainstream and specialised schools. Our contributions to this partnership is two folds: 1) We provide relevant staff (Qualified Teachers for Visual Impairments) with opportunities to be involved in the design and evaluation of novel technologies for visually impaired users 2) We contribute activities (e.g. technology demonstrations, training workshops for staff) to public events organised by the sensory consortium service |
| Collaborator Contribution | They make contributions to the research project in two ways: 1) Provide access to their staff who are Qualified Teachers for Visual Impairment who contribute as study participants 2) They provide contacts with local mainstream schools who then contribute to the research project also as partners and provide access to teachers, teaching assistants and pupils who contribute as study participants |
| Impact | - Staff from the SEN Support Services have taken part in an ethnographic scoping study - We have identified four schools through the service, and established a partnership with that school (where we are now conducting further ethnographic field studies with staff, teachers, and pupils) |
| Start Year | 2016 |
| Description | Sensory Consortium Service Berkshire |
| Organisation | Berkshire Sensory Consortium Service |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | The Sensory Consortium Service Berkshire, part of the Berkshire Council, is a service providing support for children with sensory impairments in mainstream and specialised schools. Our contributions to this partnership is two folds: 1) We provide relevant staff (Qualified Teachers for Visual Impairments) with opportunities to be involved in the design and evaluation of novel technologies for visually impaired users 2) We contribute activities (e.g. technology demonstrations, training workshops for staff) to public events organised by the sensory consortium service |
| Collaborator Contribution | They make contributions to the research project in two ways: 1) Provide access to their staff who are Qualified Teachers for Visual Impairment who contribute as study participants 2) They provide contacts with local mainstream schools who then contribute to the research project also as partners and provide access to teachers, teaching assistants and pupils who contribute as study participants |
| Impact | - Staff from the Sensory Consortium Services have taken part in an ethnographic scoping study - We have identified one school through the service so far, and established a partnership with that school (where we are now conducting further ethnographic field studies with staff, teachers, and pupils) |
| Start Year | 2016 |
| Description | Sensory Support Service Bristol |
| Organisation | Sensory Support Service |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | The Sensory Support Service, part of the Bristol City Council, is a service providing support for children with sensory impairments in mainstream and specialised schools, covering four councils in the Southwest of England. Our contributions to this partnership is two folds: 1) We provide relevant staff (Qualified Teachers for Visual Impairments) with opportunities to be involved in the design and evaluation of novel technologies for visually impaired users; 2) We contribute activities (e.g. technology demonstrations, training workshops) to public events organised by the sensory support service |
| Collaborator Contribution | They make contributions to the research project in two ways: 1) Provide access to their staff who are Qualified Teachers for Visual Impairment who contribute as study participants 2) They provide contacts with local mainstream schools who then contribute to the research project also as partners and provide access to teachers, teaching assistants and pupils who contribute as study participants |
| Impact | - Staff from the Sensory Support Services have taken part in a scoping study - We are organising a joint event this summer to demonstrate accessibility technology for visually impaired children and their parents - We have identified two schools through this service, and established partnerships with them (where we are now conducting ethnographic field studies with staff, teachers, and pupils) |
| Start Year | 2016 |
| Description | Southville Primary School, Bristol |
| Organisation | Southville Primary School |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We are currently in contact with this school, coordinating with the school's Special Education Needs Coordinator to organise discussions with staff and pupils regarding their potential involvement in the development of accessibility technology to support visually impaired children during group work |
| Collaborator Contribution | The school will provide us with access to their staff and pupils who will take part in an on going ethnographic study involving observations of teaching sessions, and interviews with teachers, staff and pupils as well as co-design workshop and technology evaluation sessions |
| Impact | Engagement with staff and pupils |
| Start Year | 2018 |
| Description | The Emmbrook School Reading |
| Organisation | The Emmbrook School |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We have visited the school, coordinated with the school's Special Education Needs Coordinator to organise discussions with staff and pupils regarding their potential involvement in the development of accessibility technology to support visually impaired children during group work |
| Collaborator Contribution | The school has provided us with access to their staff and pupils who are currently taking part in an on going ethnographic study involving observations of teaching sessions, and interviews with teachers, staff and pupils. Future activities will also involve design workshop and technology evaluation sessions |
| Impact | We were able to run an ethnographic study at this school (on going), which is helping us develop a rounded picture of the inclusion of visually impaired children in mainstream schools. This feeds directly into the process of designing technology to support inclusive learning and teaching practices |
| Start Year | 2017 |
| Description | iCASE Collaboration (Microsoft) |
| Organisation | Microsoft Research |
| Department | Microsoft Research Cambridge |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Joint EPSRC iCASE award |
| Collaborator Contribution | Joint EPSRC iCASE award |
| Impact | One paper under review |
| Start Year | 2020 |
| Description | iCASE Collaboration (Microsoft) |
| Organisation | Microsoft Research |
| Department | Microsoft Research Cambridge |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | Joint EPSRC iCASE award |
| Collaborator Contribution | Joint EPSRC iCASE award |
| Impact | One paper under review |
| Start Year | 2020 |
| Title | Inclusive Robot-based Game |
| Description | A robot-based game using Ozobots and a set of low-fidelity materials. The game allows children with mixed visual ability to to engage in map-reading activities, tangible interaction and crafting with robots, and collaborative mobility within school premises |
| Type Of Technology | Physical Model/Kit |
| Year Produced | 2019 |
| Impact | We have evaluated the game in-situ in an inclusive schools and demonstrated how children with and without visual impairment can be supported in engaging in inclusive social play |
| Title | Multisensory Collaborative Story Mapping Tool |
| Description | A system resulting from the co-design sessions with visually impaired and sighted children and their teachers, is a multisensory story mapping prototype for facilitating storytelling tasks (including mapping, sequencing, composition and performance) with groups of primary-aged children with mixed visual abilities. The system is comprised of four main components; i) an audio sampler and playback unit; ii) a grid for organising the narrative structure; iii) a character module for exploring the narrative; and iv) a scene module, which introduces further multisensory elements to the story, including light and scent display. |
| Type Of Technology | Software |
| Year Produced | 2018 |
| Impact | The tool has been used at a partner school, and departs from the way storytelling is practice with primary aged children. Traditional method involves low-fi crafting and adaptation of stories including addition of Braille notation. Importantly, this traditional method does not engage both visually impaired and sighted children, making storytelling essentially exclusive to either groups separately. This tool allows teachers to engage both children with and without visual impairments in storytelling, which helps make this learning activity more inclusive. |
| Title | Voxtopus: Multisensory Voice-Interface for Inclusive Peer Revision |
| Description | a Voice-based User Interface application using Amazon Alexa named Voxtopus. Voxtopus allows groups of pupils to engage in peer revision and augments the Echo device with a set of physical controllers that support audio-tactile display and can be used for answering and navigating questions. Importantly, Voxtopus supports potential for collaborative learning activities, where pupils participate in the design of personalised revision materials (summaries and MCQ quizzes) and share them with others. The technology therefore enables and supports inclusive pedagogy and activities. |
| Type Of Technology | Software |
| Year Produced | 2018 |
| Open Source License? | Yes |
| Impact | We are currently in the process of finalising and deploying Voxtopus for a longitudinal evaluation at a school, where it will be accessible to all pupils from the resources room in a space dubbed the "Voice Booth". The tool allows for promotion of inclusive peer revision between visually impaired and sighted children. |
| Description | Co-desig workshops |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | Part of the research activities associated with this grant is to engage with local schools to conduct participatory design activities and evaluations. In additional to previous schools we engaged with, we engaged with two further schools through participatory design workshops with children and educational staff (including teachers, teachings assistants and trainees), which is leading to the shaping of the kinds of technologies we are developing as part of this project, as well extreme interest from the staff and pupils in the project and the alternative educational technologies we are developing. These activities regularly lead to the planning of a number of follow up design activities, which ensures continuous engagement with participants |
| Year(s) Of Engagement Activity | 2019 |
| Description | Inclusive Hackathon |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Schools |
| Results and Impact | A one day hackathon event bringing together hackers from a wide ranges of areas together with actual end-users to hack inclusive education technologies. The event is unique in so far as it include activities that engages programmers, education students, product designers, qualified teachers for visual impairments, and children with and without visual impairments. The hackathon event mixes hacking with inclusive co-design to conceptualise and prototype education technologies that are inclusive of both visually impaired and sighted children. The event has a particular focus on voice-based and multisensory technologies. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Inclusive Technologies Special Interest Group |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | A Special Interest Group meeting at ACM SIGCHI 2019 Conference where ~20 experts in the design accessible technologies to discuss emerging opportunities for inclusive technologies for people with visual impairments |
| Year(s) Of Engagement Activity | 2019 |
| Description | Inclusive Technologies Workshop |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | A workshop at ACM SIGCHI 2018 Conference where 20 experts in the design accessible technologies to discuss emerging opportunities for inclusive technologies for people with visual impairments. The plan is to create a community of researchers around this topic and to engage with practitioners and professionals to define the directions for this new area of research interests |
| Year(s) Of Engagement Activity | 2018 |
| Description | Inclusive Technologies Workshop 2 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | A workshop at the ACM ICMI conference, following on in the series of workshops and special interest group meeting from previous years, with the broader aim of creating a community of researchers and practitioners to define and shape research and practice in the area of inclusive technologies |
| Year(s) Of Engagement Activity | 2020 |
| Description | Inclusive Technologies for marginalised children |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | A workshop at the ACM IDC conference on interaction design and children, following on in the series of workshops and special interest group meeting from previous years, with the broader aim of creating a community of researchers and practitioners to define and shape research and practice in the area of inclusive technologies |
| Year(s) Of Engagement Activity | 2020 |
| Description | Participatory design workshops |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | Part of the research activities associated with this grant is to engage with local schools to conduct participatory design activities and evaluations. In additional to previous schools we engaged with, we engaged with two further schools through participatory design workshops with children and educational staff (including teachers, teachings assistants and trainees), which is leading to the shaping of the kinds of technologies we are developing as part of this project, as well extreme interest from the staff and pupils in the project and the alternative educational technologies we are developing. These activities regularly lead to the planning of a number of follow up design activities, which ensures continuous engagement with participants |
| Year(s) Of Engagement Activity | 2017,2018 |
| Description | School visit |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | Part of the research activities associated with this grant is to engage with local schools to conduct ethnographic studies, technology design activities, and evaluations. So far, we have visited four schools; one primary and three secondary. In all cases the visit included discussions with staff and pupils about the research activities, which often led to extreme interest from the staff and pupils in being involved in the project, and led to the planning of a number of follow up activities, including organising technology design activities and workshops |
| Year(s) Of Engagement Activity | 2016,2017 |
| Description | Tool exhibition at We The Curious, Bristol |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Public/other audiences |
| Results and Impact | An exhibit at We the Curious museum show case a voice-based tool for inclusive education augmented with multisensory input and output. The tool allows up to three user to play Q/A quizzes using a set of physical controllers to interact with an Amazon Echo. The purpose of this exhibition was to expose the general public to the latest outputs of the CRITICAL project, it sparked audience interest in terms of potential future was to interact with voice-based technology through other sensory modalities. |
| Year(s) Of Engagement Activity | 2018 |