Quantifying bilingual experience (Q-BEx): optimising tools for educators, clinicians and researchers

Lead Research Organisation: University of Leeds
Department Name: Sch of Languages, Cultures and Societies

Abstract

At least a fifth of primary school children in the UK are bilingual: they grow up with two or more languages. Bilingualism is a linguistic, cultural and economic asset, but it also poses practical challenges for professionals having to deal with an extremely diverse group of children. For instance, it is difficult to evaluate whether a bilingual child's progress in the school language is due to limited language experience (e.g. at home) compared with monolingual peers, or whether it indicates a disorder. Teachers need to know when they can expect bilingual children to have "caught up" with monolinguals in order to inform their assessment of their progress.

One of the strongest determinants of children's language development is the amount of time they spend hearing and speaking the language. Measuring this language experience is therefore essential to inform the expectations of teachers and speech and language therapists (SLTs) regarding children's knowledge of the school language. For example, a child who is only exposed to English at school is likely to be less proficient than a child who also hears English from a parent or siblings. This can profoundly affect many aspects of that child's educational development.

Researchers have recently started to address the need to quantify the bilingual language experience through the creation of parental questionnaires about the languages spoken to and by the child with different people (parents, siblings, etc.), in different settings (home, school, etc.), and during different activities (reading, watching TV, etc.). Because the questionnaires have been developed by independent teams of researchers, the information they elicit varies - sometimes substantially, resulting in a lack of comparability across studies. Some questionnaires have been used exclusively for research purposes and their length and complexity make them difficult to use by teachers and SLTs, who are often pressed for time and may deal with parents with low levels of proficiency in the community language. Another major challenge is the typically low completion rates of parental questionnaires.

Q-BEx will bring a step-change in the measurement of bilingual language experience.

* It is the first, multidisciplinary endeavour to establish an optimal metric of bilingual language experience based on a consensus among researchers, SLTs and educators on what aspects of language experience to index, as well as an in-depth review of existing tools.

* It will for the first time deliver user-friendly, online questionnaires (and their associated back-end calculators) to return measures of current and cumulative language experience in real time. The questionnaires will be available in 13 languages, and vary in length and level of detail: the shortest version will be useful when parental consultation is challenging; the longest version will yield more fine-grained measures to enable SLTs and researchers to carry out in-depth enquiries.

* Q-BEx will be the first to exploit advanced quantitative methods to identify the right level of questionnaire detail (to meet the needs of end-users) and to assess the reliability of the resulting measures as predictors of language proficiency. Reliability and cross-language validity will be assessed using new data from 300 children in 3 different countries. Based on this assessment, Q-BEx will provide evidence-based guidance to inform users' choice on the level of questionnaire detail most appropriate to their needs.

* Q-BEx will develop an objective method to identify early those bilingual children in need of support with their school language, thereby addressing the "catch up" question.

* Q-BEx will contribute to advancing bilingualism research by exploiting cutting-edge statistical techniques, and making all scripts available to facilitate replication.

Planned Impact

*IMPACT DELIVERABLES:

Q-BEx will
1. develop user-friendly online calculators to measure bilingual children's current and cumulative experience in each language, for teachers and speech and language therapists (SLTs) to use when creating a language profile of the children in their care;
2. create questionnaires to collect data on bilingual children's exposure and use of each language, to be used in conjunction with the online calculators. The questionnaires will be graded in detail and translated in twelve of the most frequent additional languages in Western Europe and North America;
3. provide reliability assessment of the calculators' output as predictor of language proficiency based on new data from 300 children;
4. develop an objective method that uses the relationship between language exposure and proficiency to identify early those bilingual children in need of support with their school language, thereby addressing the "catch up" question.

*IMPACT BENEFICIARIES:

The principal beneficiaries of the impact goals are teachers and SLTs in the UK, the Netherlands, and France. Representatives from the two non-academic stakeholder groups (teachers and SLTs) have been consulted on the appropriateness and relevance of the objectives and deliverables. An initial online survey between March and December 2017 (https://www.surveymonkey.co.uk/r/D7Y8JSJ) targeting practitioners who routinely need to assess bilingual children has revealed that only 2% (out of 131) are satisfied with the tools and guidance at their disposal to assess bilingual children. Their concerns about the language assessment of bilingual children have fed directly into this proposal. Continued engagement with these groups will be essential to the success of Q-BEx.

Teachers will directly benefit from a tool to better understand the linguistic background of the bilingual children in their classroom. In England and Wales teachers have to comply with a newly introduced assessment scheme introduced by the Department for Education in 2016: Stages of Proficiency. The aim is to gather a better picture of the linguistic strengths of children with English as an Additional Language and to monitor their progress. Conducting the assessment in conjunction with information on children's cumulative and current language exposure will provide a more holistic picture of the child's bilingual profile, and will allow teachers to make predictions about children's longitudinal linguistic trajectories.
In the first instance teachers will be contacted through their national professional associations (NALDIC in the UK: https://naldic.org.uk) for the purpose of stakeholder involvement and for taking part in knowledge transfer activities. Serratrice (Co-I) has established links with a number of NALDIC Regional Interest Groups across England through the Institute of Education at the University of Reading and through Bilingualism Matters@Reading.

SLTs need help with the holistic assessment of bilingual children in two languages as recommended by the RCSLT. There is currently no approved protocol to take a case history including details on the child's exposure to their two (or more) languages. Data from our scoping survey suggest that SLTs would welcome more practical guidance on what to include in the parental interview that would allow them to build a more comprehensive picture of how children are exposed and use their two languages.
SLTs will be contacted via the Royal College of Speech and Language Therapists (RCSLT). Serratrice (Co-I) has strong connections with the RCLST through previous collaborations (e.g. reviewing the RCLST's guidance for Gaelic speakers in 2015; reviewing the RCSLT's guidance on assessment and intervention with bilingual speakers), and through the SLT programme at the University of Reading where she currently teaches.

Publications

10 25 50
 
Description We have carried out an international consensus survey among professionals working with bilingual children, to identify which aspects of their bilingual experience needs to be documented (to inform their work supporting or studying these children). A total of 132 professionals from across 30 countries participated in this two-round survey, including researchers, teachers and speech & language therapists.

The survey results indicate substantial agreement about the need for a set of common measures of children's bilingual language experience, to allow comparability
across the studies and to facilitate communication across sectors (agreement rate: 96%). There is also strong agreement that these measures need to be applicable to children who speak more than two languages (agreement rate: 90%), and that the questionnaire should be accompanied by a tool yielding automatic calculation of objective scores in each language of current and cumulative language exposure and use (agreement rate for each aspect: 90% or more). Panellists also agreed that, where possible, the
questionnaire should provide a clear guidance about how to interpret the data it generates (agreement rate: 93%), and that it should include a section to identify children who might be at risk for a Developmental Language Disorder (agreement rate: 77%)
Exploitation Route The results of this survey have been used to inform the design of a new online tool to document and quantify children's bilingual experience. The tool's Beta version is now live and its use is being piloted by three international research teams external to our project. A publication reporting on the survey is now available as an OSF preprint, together with the data (https://osf.io/2pd65/).
Sectors Communities and Social Services/Policy,Education

URL https://q-bex.org/project-milestones/delphi-consensus-survey/
 
Title Online questionnaire and back-end calculator to quantify children's bilingualism or trilingualism. 
Description We have created a user-friendly, online questionnaire (and back-end calculators) to return a number of measures relevant for developing language profiles of bilingual (or trilingual) children. The questionnaire is designed to be used by researchers, teachers, or speech & language therapists. The questionnaire can be customised in many ways to facilitate administration and provide the required level of detail required by the professional. We have already translated it in 5 other languages (French, Dutch, German, Japanese, Spanish) and are working on the translation into a further 10 languages. 
Type Of Technology Webtool/Application 
Year Produced 2021 
Impact The tool is already used by a number of international researchers. A full report of impact will be added at the end of the project. 
URL https://www.q-bex.org/
 
Description Autism and Bilingualism: Practitioner Webinar, University of Edinburgh 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact In this talk, Drasko Kascelan presented the creation of the Q-BEx questionnaire as a collaborative project with researchers, teachers and speech % language therapists (SLTs). The presentation discussed the usability of the questionnaire in the SLT practice, particularly in relation to the risk assessment questions.
Year(s) Of Engagement Activity 2021
URL https://patrickwildcentre.com/autism-bilingualism-practitioner-webinar/
 
Description Bilingualism Matters Research Symposium - Invited Keynote 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The keynote was focused on raising awareness about the importance of research on language diversity and on the documentation of the multilingual language experience.
Year(s) Of Engagement Activity 2022
URL https://www.bilingualism-matters.org/events
 
Description Growing Up Bilingual - British Psychological Society - Researchers-Practitioners workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Researcher-practitioner workshop on different aspects of Children Growing up Bilingual - This was a hybrid event with 40 people in attendance in person and another 30-40 attendees online. A part of her presentation, Prof. Serratrice - one of the Co-Is - introduced the Q-BEx questionnaire as the new tool to measure the bilingual language experience. Plans are in progress for a meeting in June-July with academics and practitioners with a view to discussing grant applications including Q-BEx.
Year(s) Of Engagement Activity 2022
URL https://www.eventbrite.co.uk/e/growing-up-bilingual-researcher-and-practitioner-synergies-tickets-24...
 
Description Invited talk at the "Talk To Your Baby" conference organised by the National Literacy Trust. The talk was entitled "Supporting bilingual children". 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Cecile De Cat (PI) was invited to address the TTYB conference audience to present the state-of-the-art in supporting multilingual children in Early Years settings. The aim of the presentation was to convince Early Years professionals that to support multilingual children, it is necessary to understand the diversity of their language experience. The first part of the talk presented research evidence in support of this view, and the second part introduced the Q-BEx tool and showed how it can inform the work of Early Years practitioners working with multilingual children.

There were 150 attendees, including:
- Speech and language therapists
- Early years/nursery practitioners
- Local authority children's services senior managers
- Literacy specialists
- Primary teachers
- Academics (in education and early development)
- senior managers in third sector organisations

Feedback from the organisers and audience was very positive. A number of professionals got in touch afterwards by email to take things further. For example, a Family Support Manager from a charity supporting deaf children across the country is now considering using Q-BEx as a tool to collect information on home languages spoken and family heritage from the large percentage of multilingual families they work with.
As another example, a Speech and Language Therapist working on the Welsh Government's "Talk with Me campaign" got in touch to discuss how improve the support for 2-year-olds from lower SES backgrounds who are from (often) English speaking homes (many without language rich environments) and who attend Welsh language immersion Flying Start childcare settings.
Year(s) Of Engagement Activity 2023
URL https://literacytrust.org.uk/news/hosting-talk-to-your-baby-2023-in-birmingham/
 
Description Keynote presentation - Researcher and Practitioner Synergies Growing Up Bilingual 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact A workshop bringing together teachers, charities and academics working on bilingualism and the quantification of language experience
Year(s) Of Engagement Activity 2022
URL https://www.bps.org.uk/psychologist/beginnings-bilingual-bridges
 
Description Online meeting with Lewisham and Greenwich NHS Trust Speech and Language Therapist Service Manager 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Meeting with the Lewisham and Greenwich NHS Trust Speech and Language Therapist (SLT) Service Manager to involve the SLTs working with multilingual children in the piloting of the Q-BEx questionnaire.
Year(s) Of Engagement Activity 2022
 
Description Online practitioner training _ Study Day for the Royal College of Speech and Language Therapists Centre for Excellence Network on Bilingualism 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A dedicated study day on the use of the Q-BEx in clinical practice
Year(s) Of Engagement Activity 2022
 
Description Presentation at the BeCurious festival 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact This talk was delivered by Drasko Kascelan, as part of a series of presentations about research conducted at the University of Leeds. The theme of the session was Inclusion and Identity. Project's work was presented by discussing the diversity of bilingual profiles when quantifying language experience in children, and how our questionnaire can help in tapping into various linguistic backgrounds.
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=6ff0tyaJ6tg&list=PLjEqI4wfi6ycaRkZ2E4fZFuNtFmEu3VDz&index=3
 
Description Presentation of the Q-BEx questionnaire - Princethorpe Junior School Birmingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Presentation of the rationale for the Q-BEx questionnaire to teachers and parents of bilingual children and hands on demonstration of how to use the questionnaire and generate reports.
Year(s) Of Engagement Activity 2022
 
Description Presentation of the Q-BEx questionnaire - Szkola Polska przy ambasadzie RP w Londynie 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Presentation of the rationale for the Q-BEx questionnaire to teachers and parents of bilingual children and hands on demonstration of how to use the questionnaire and generate reports.
Year(s) Of Engagement Activity 2022
 
Description Presentation of the Q-BEx questionnaire St Raphael's RC Primary - London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Presentation of the rationale for the Q-BEx questionnaire to teachers and parents of bilingual children and hands on demonstration of how to use the questionnaire and generate reports.
Year(s) Of Engagement Activity 2022
 
Description Presentation of the Q-BEx questionnaire to teachers and parents at Oldfield Primary - London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Presentation of the rationale for the Q-BEx questionnaire to teachers and parents of bilingual children and hands on demonstration of how to use the questionnaire and generate reports.
Year(s) Of Engagement Activity 2022
 
Description Presentation to the Special Interest Group for SLTs in Ireland interested in Linguistic & Cultural Diversity 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This is an event in which the PI - Prof. De Cat, the UK-based co-I Prof. Serratrice and the UK-based Research Assistant Mr Peter Gillam presented Q-BEx to the Special Interest Group in Linguistic and Cultural Diversity - a professional group of speech and language therapists working with multilingual children. The intention is to form a partnership in which the SLTs will pilot the questionnaire as part of their clinical practice and feed back to the research team to further adjust the questionnaire for accessibility.
Year(s) Of Engagement Activity 2022
URL https://twitter.com/LCDSIG_IASLT/status/1493610029957722116
 
Description Questionnaire trial and feedback from 6 primary and secondary schools 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact The Q-BEx questionnaire was introduced to six primary and secondary schools with a view to adopting it as part of their school's EAL policy
Year(s) Of Engagement Activity 2022
 
Description School-Centred Initial Teacher Training session (Bradford) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact We delivered a 2-hour training session via the Bradford Birth-to-19 SCITT (school-centred initial teacher training) on 17/01/2022, entitled "Diversity among EAL pupils: challenges and opportunities". The aim was to increase trainees' awareness of the need to know the language background of their bi/multilingual pupils, in order to be able to identify their support needs. We demonstration how to use the Q-BEx tool and how to use the output. Participant feedback showed that the session was perceived as interesting, informative and relevant. They also indicated that what they had learned would influence their practice.
Year(s) Of Engagement Activity 2022
 
Description Visit to the Polish Language Complementary School in Slough 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Visit to the Polish Language Complementary School in Slough as part of local community outreach - presentation of the Q-BEx questionnaire to teachers and parents.
Year(s) Of Engagement Activity 2022
 
Description Webinar - Do kids get confused when you talk to them in several languages? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact A webinar to raise awareness about the role of children's language experience in code-switching - 438 views on 6/3/2023
Year(s) Of Engagement Activity 2022
URL https://www.youtube.com/watch?v=__6Qh-b502U
 
Description Webinar - The Multilingual Garden - Bilingualism and the pandemic: unexpected linguistic advantages 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact A talk on the relationship between language exposure and language development in bilingual children during the Covid-19 pandemic. Raising awareness of the importance of maintaining the heritage language in multilingual families
Year(s) Of Engagement Activity 2022
URL https://www.youtube.com/watch?v=8gFJ3JbLZbQ&t=508s
 
Description Workshop for teachers, delivered at the annual meeting of the NALDIC (the national subject association for teachers of English as an Additional Language) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The workshop included:

"Reflection on practice - how do you assess EAL pupils' language proficiency and experience"?
Interactive online activities for live participation
Hands-on experience of a new tool to profile EAL pupils
Opportunity for co-creation of the school-optimised version of the tool

Discussion of teachers' current practice (10 minutes) | (Based around interactive online poll)
- What language assessment of EAL students do you routinely use in your context?
- What are the issues with assessing language proficiency and experience of EAL students in your context?
- What background information are you given about your EAL students and how do you use this?
- Why is it important to understand experience as well as proficiency?

Introduction of Q-BEx project (5 minutes)
- Aims of the project
- Why is there a need for this?
- How can this be useful for you in your school? What solutions might it provide?

A closer look at the Q-BEx questionnaire (10 minutes)
- Discussion over meaning of the 7 modules: What information can we elicit from the different modules? Which parts would be interesting and useful for you and your school?
- Demonstration of key features of the questionnaire (online)

Interactive use of the questionnaire (15-20 minutes)
- Use of case study so that teachers can engage with the tool using their own device (mobile, tablet, computer)- different tasks/activities to try

Group discussion: (5 minutes) | (Based around interactive online poll)
- Having looked at the tool, which children would you find this most useful for? E.g. all children, new arrivals, start of primary/secondary school?

Co-creation of school-optimised version - group discussion: (5 minutes)
- Opportunity to contribute to the streamlined version for schools
- Which aspects are most relevant for your practice?
- What format do you think would work best for you?
- What format of output reporting would be best for you?

Questions (5 minutes)
Year(s) Of Engagement Activity 2021
URL https://naldic.org.uk/professional-learning-cpd/annual-conference/naldic-29-conference-2021/#worksho...