SPeech Across Dialects of English (SPADE): large-scale digital analysis of a spoken language across space and time

Lead Research Organisation: University of Glasgow
Department Name: School of Critical Studies

Abstract

Obtaining a data visualization of a text search within seconds via generic, large-scale search algorithms, such as Google n-gram viewer, is available to anyone. By contrast, speech research is only now entering its own 'big data' revolution. Historically, linguistic research has tended to carry out fine-grained analysis of a few aspects of speech from one or a few languages or dialects. The current scale of speech research studies has shaped our understanding of spoken language and the kinds of questions that we ask. Today, massive digital collections of transcribed speech are available from many different languages, gathered for many different purposes: from oral histories, to large datasets for training speech recognition systems, to legal and political interactions. Sophisticated speech processing tools exist to analyze these data, but require substantial technical skill. Given this confluence of data and tools, linguists have a new opportunity to answer fundamental questions about the nature and development of spoken language. Our project seeks to establish the key tools to enable large-scale speech research to become as powerful and pervasive as large-scale text mining. It is based on a partnership of three teams based in Scotland, Canada and the US. Together we will exploit methods from computing science and put them to work with tools and methods from speech science, linguistics and digital humanities, to discover how much the sounds of English across the Atlantic vary over space and time.

We will develop an innovative and user-friendly software which exploits the availability of existing speech data and speech processing tools to facilitate large-scale integrated speech corpus analysis across many datasets together. The gains of such an approach are substantial: linguists will be able to scale up answers to existing research questions from one to many varieties of a language, and ask new and different questions about spoken language within and across social, regional, and cultural, contexts. Computational linguistics, speech technology, forensic and clinical linguistics researchers, who engage with variability in spoken language, will also benefit directly from our software. This project will also open up vast potential for those who already use digital scholarship for spoken language collections in the humanities and social sciences more broadly, e.g. literary scholars, sociologists, anthropologists, historians, political scientists. The possibility of ethically non-invasive inspection of speech and texts will allow analysts to uncover far more than is possible through textual analysis alone.

Our project will develop and apply our new software to a global language, English, using 43 existing public and private spoken datasets of Old World (British Isles) and New World (North American) English, across an effective time span of more than 100 years, spanning the entire 20th century. Much of what we know about spoken English comes from influential studies on a few specific aspects of speech from one or two dialects. This vast literature has established important research questions which can be investigated for the first time on a much larger scale, through standardized data across many different varieties of English. Our large-scale study will complement current-scale studies, by enabling us to consider stability and change in English across the 20th century on an unparalleled scale. The global nature of English means that our findings will be interesting and relevant to a large international non-academic audience; they will be made accessible through an innovative and dynamic visualization of linguistic variation via an interactive sound mapping website. In addition to new insights into spoken English, this project will also lay the crucial groundwork for large-scale speech studies across many datasets from different languages, of different formats and structures.

Planned Impact

Who will benefit from this research?
The global nature of English means that our findings about Cross-Atlantic English speech across the 20th century will be interesting to a huge, international, non-academic audience, within and well beyond the sponsor countries. Our research tools and findings will be interesting to those working professionally with language and English at different levels, e.g. teachers and students, in industry those working with speech synthesis and recognition, in forensic speech practice, and clinical practice. More broadly, our spoken analysis tool has substantial potential for those working with spoken language in museums, informatics, libraries and schools, as well as the interested general public.

How will they benefit from this research?
SPADE make it possible to search spoken language in the same way as written texts, but without the need to listen to any speech whilst doing so. We aim to develop our spoken language analysis software and apply it to Old and New World (North American Englishes). our immediate findings will be useful and interesting to non-academic users of many kinds, but especially those who work with English. We will also make our source code publically accessible as we develop it, enabling all, including non-academic users, to test and use it for themselves. We will also develop an open access web resource, to make our results accessible during and towards the end of the project. In the longer term, SPADE will enable the ethically non-invasive inspection of speech and texts, and hence will allow analysts of all kinds, not just academics, to uncover far more about their spoken materials than is possible through textual analysis alone.

What will be done to ensure that they have the opportunity to benefit from this activity?
Our main specific outreach activity for SPADE is the creation of a digital media product, through which we will visualise both the spread of sounds and sound changes across the span of the Cross-Atlantic English varieties, and also the results themselves. With present technology we envisage creating a website with interactive (synthesized) sound examples, Mapping the dynamics of English. The large-scale, comparative nature of SPADE means that we can represent places not just with single examples, but with a continuous range of speech variants observed across time and social space at a particular location on the map, offering an innovative, dynamic, view of linguistic variation and change for English. The website will be designed with a responsive interface to allow easy access for fixed computers and mobile devices, and will be built and maintained indefinitely at Glasgow, given the strong expertise and experience within the GlasgowU Digital Humanities Research Network and GULP. Equally important, throughout the duration of the project, SPADE will ensure a lively and interactive relationship with the international public, through updates, announcements and activities via social and digital media (e.g. website, Twitter, Instagram, WordPress blog), and local, national and international public engagement. All members of the project team expect to take part in public engagement events, public talks, and media releases publicising the stages of the research project. We will also prepare and release our software as open-source with full documentation. We will work with both academic, but also skilled non-academic users, to ensure that we develop interfaces which are user friendly for a large range of users. The workshop in the final year of the project at a key Digital Humanities conference will ensure that we communicate effectively not only academic users, but also non-academic beneficiaries wishing to work with, and search through, spoken language corpora.

Publications

10 25 50
 
Description The SPADE project aims to develop accessible speech corpus analysis tools, and then to use these tools to investigate Old and New World Englishes across time (the course of the 20th century) and space (UK and Ireland; US and Canada). To date the project has been running for 19 months. Since March 2018, we have: (1) developed and released the first version of the Integrated Speech Corpus ANalysis software, ISCAN v 0.1, which was tested within the project and had its first public outing at the NWAV ISCAN workshop in October 2018. ISCAN has now been installed on several computers at McGill, and its first remote location, in Glasgow this spring, and team-members at Oregon have also been also to successfully worked remotely with it on McGill server. The ISCAN software has two instantiations for users, 'under the hood' via scripts, or through the ISCAN GUI, Graphical User Interface, which enables less experienced users to search, analyse and query multiple corpora. Substantial software development includes: shift to a server-client architecture; work on scaling up and optimisation of corpus importers and analytical procedures; documentation. (2) we have now developed fast, consistent, acoustic analytical tools for the analysis of vowels, sibilants and stops, specifically: vowel formants, sibilant spectral characteristics, segment durations, Voice Onset Time for stops, with dynamic (tracks) and static (single point) measures. (3) implemented our data collection using our novel GDPR-compliant Data Transfer Agreement, to make substantial progress with collection of Phase 2 privately held datasets across all three countries - to date 20 of the 36 UK datasets have been collected. (4) used the new algorithms to import and analyse 9 speech corpora totalling around 450 speakers for 6 English dialects, for subprojects on vowels and sibilants, comprising the first, large-scale analyses of these sound categories across such a large number of speakers and dialects of English; (5) initiate new subprojects on Scottish Vowel Length Rule, vowel duration, and VOT; (6) presented 2 posters (LabPhon16, Lisbon, June18), 5 talks (NWAV47, NY, Oct18; Huddersfield, Aug18; Newcastle, Nov18); 3 written paper acceptances for publication and oral presentation (ICPhS2019, Melbourne, Aug19); 1 poster acceptances (WSC5, June19, Davis California), 1 paper acceptance (BICLCE2019, Sep19, Bamburg); invited chapter on on data management for SPADE/ISCAN submitted for the MIT Open Handbook of Linguistic Data Management.
Exploitation Route Our software for analyzing speech corpora will be ultimately useful to anyone wishing to analyse aspects of spoken language, especially recordings which may be private for ethical reasons, so for those working in the law, digital./communication/IT, education, and healthcare. More specifically during the lifetime of the project, we anticipate those working on/with spoken language in different ways to benefit from our methodological developments and research findings.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Healthcare,Government, Democracy and Justice,Culture, Heritage, Museums and Collections

URL https://spade.glasgow.ac.uk/
 
Description We have continued to give presentations on SPADE, the project, the methods, tools and approach at a range of workshops, events and colloquia. These include both academic and non-academic audiences, including e.g. Oxford University Press Dictionary team, Amazon Tech, forensic phoneticians and caseworkers, amongst others.
First Year Of Impact 2018
Sector Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice,Security and Diplomacy
Impact Types Societal

 
Title ISCAN (Integrated Speech Corpus ANalysis) software v 0.1 
Description A key methodological output of the SPADE project is the development of freely-accessible, integrated speech corpus analysis software, which produces consistent acoustic phonetic measures from multiple spoken language corpora of diverse formats. The first version of ISCAN was released in 30 July 2018. It was extensively tested by the project team until October, when a public tutorial enabling users to carry out a set of tutorial routines took place in October 2018. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact ISCAN software was introduced to sociolinguists and phoneticians at the NWAV47 conference special workshop, 18 October 2018. The software was greeted with substantial enthusiasm, given the quality of the acoustic analysis, the speed of analysis across large speech corpora, and the fact that it can work with multiple diverse corpus formats. Members of the workshop requested an extension to languages other than English - ISCAN is not a language-specific tool, but we are currently working to ensure analyses (e.g. vowel analyses) can be carried out irrespective of language heritage. 
URL https://github.com/MontrealCorpusTools/ISCAN
 
Description SPADE at BAAP 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Presentation of first analysis of /s/ results at the BAAP 2018 Colloquium. A number of people have now offered to allow us to work with their speech datasets.
Year(s) Of Engagement Activity 2017
URL https://blogs.kent.ac.uk/baap/talks-day-2/
 
Description SPADE at CLiS 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Digging into speech corpora: Introducing SPADE, a new initiative for mining spoken datasets on a large scale is the keynote presentation at the Corpus Linguistics in Scotland event Corpus linguistics and cross-disciplinarity in Glasgow.
Year(s) Of Engagement Activity 2017
URL https://cdn.evbuc.com/eventlogos/175853256/schedule.jpg
 
Description SPADE at NWAV46 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact SPADE is included in the workshop Sociolinguistics and forensic speech science: knowledge- and data-sharing at the New Ways of Analyzing Variation 46 conference in Madison, Wisconsin.
Year(s) Of Engagement Activity 2017
URL https://dept.english.wisc.edu/nwav46/wp-content/uploads/2016/09/NWAV-46-Booklet-Nov3.pdf
 
Description SPADE at the 174th Meeting of the Acoustical Society of America 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact SPADE was included in the presentation Sociophonetic trends in studies of Southern English at the 174th meeting of the Acoustical Society of America in New Orleans.
Year(s) Of Engagement Activity 2017
URL http://acousticalsociety.org/sites/default/files/fullweek.pdf
 
Description interview for 'accents' programme as part of BBC Radio 4's Broadcasting House; broadcast 18 November 2018 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Personal invitation to contribute to a segment on 'accents, variation and change' for BBC Radio 4's Broadcasting House programme, which was broadcast on Sunday 18 November 2018. The programme producer, Dearbhail Starr, was fascinated by the material which I contributed, which covered phonetic variation and change in Scottish English, based on research relating to the articulatory phonetic investigation of speech, and large-scale analysis of speech variation across English accents.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/programmes/m00018sg
 
Description poster presentation on /s/-retraction across English, LabPhon16, University of Lisbon, 19-22 June 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Poster presentation of 'Dialectal and social factors affect the phonetic bases of English /s/-retraction', authors: Jane Stuart-Smith, Morgan Sonderegger, Michael McAuliffe, Rachel Macdonald, Jeff Mielke, Erik Thomas and Robin Dodsworth, at 16th Conference on Laboratory Phonology - LabPhon16, held at University of Lisbon, 19-22 June 2018. First presentation of SPADE research on /s/-retraction, from 400+ speakers, to the international laboratory phonology community. Great deal of interest in both the research, but also our new Integrated Speech Corpus ANalysis software, ISCAN. Led to offers of more datasets for the SPADE project.
Year(s) Of Engagement Activity 2018
 
Description poster presentation on vowel variation across English, LabPhon16, University of Lisbon, 19-22 June 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Poster presentation of 'Age vectors vs. axes of intraspeaker variation for North American
and Scottish English vowel formants', authors: Erik Thomas, Jeff Mielke, Josef Fruehwald, Jordan Holley, Michael
McAuliffe, Morgan Sonderegger, Jane Stuart-Smith, Robin Dodsworth and Tyler
Kendall, at 16th Conference on Laboratory Phonology - LabPhon16, held at University of Lisbon, 19-22 June 2018. First presentation of SPADE research on vowel spaces and vowel trajectories in varieties of English, from 400+ speakers, to the international laboratory phonology community. Great deal of interest in both the research, but also our new Integrated Speech Corpus ANalysis software, ISCAN. Led to offers of more datasets for the SPADE project.
Year(s) Of Engagement Activity 2018
 
Description talk on /s/-retraction across English, at NWAV47, New York University, 18-21 October 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk on 'Dialectal and social factors affect the phonetic bases of English /s/-retraction', authors: Jane Stuart-Smith, Morgan Sonderegger, Michael McAuliffe, Rachel Macdonald, Jeff Mielke, Erik Thomas and Robin Dodsworth, at NWAV47, New York University, 18-21 October 2018. First presentation of SPADE research on /s/-retraction, from 400+ speakers, to the international sociolinguistics community. Great deal of interest in both the research, but also our new Integrated Speech Corpus ANalysis software, ISCAN. Led to offers of more datasets for the SPADE project.
Year(s) Of Engagement Activity 2018
URL https://wp.nyu.edu/nwav47/
 
Description talk on ISCAN software for SPADE, at Linguistics Seminar, 8 November 2018, University of Newcastle 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Stuart-Smith was invited to give a talk on the ISCAN (Integrated Speech Corpus ANalysis) software for SPADE, at Linguistics Seminar, 8 November 2018, University of Newcastle. The Newcastle speech group has a strong emphasis on Arabic phonetics, so the talk led to discussion about how to create a similar kind of cross-Arabic speech project. The ISCAN software can be used for any language, not only English - the focus on English for SPADE relates to the need for a specific focus for this first 3-year project.
Year(s) Of Engagement Activity 2018
 
Description talk on ISCAN software for SPADE, at the WYRED project Data Sharing Event, 2 August 2018, University of Huddersfield 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The SPADE project was the key focus of the talk, 'Developing tools for data sharing: Polyglot + ISCAN (Integrated Speech Corpus ANalysis)' given by Stuart-Smith at the WYRED project Data Sharing Event held on 2 August 2018, at the University of Huddersfield. This was a satellite event of the 27th Annual Conference of the International Association for Forensic Phonetics and Acoustics (IAFPA). Participants ranged from practising forensic phoneticians to those working in the forensic and general speech technology industry. Very great interest was shown in the project, both research aims, but also ISCAN software and its potential for mining spoken language corpora, also for forensic purposes, without the need for visually inspecting or hearing speech, and hence very useful for ethically-restricted corpora such as police interviews.
Year(s) Of Engagement Activity 2018
URL http://wyredproject.co.uk/data-sharing-satellite-event/
 
Description talk on vowel variation across English, at NWAV47, New York University, 18-21 October 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk on 'Age vectors vs. axes of intraspeaker variation for North American and Scottish English vowel formants', authors: Jeff Mielke, Josef Fruehwald, Erik Thomas, Michael McAuliffe, Morgan Sonderegger and Robin Dodsworth, at NWAV47, New York University, 18-21 October 2018. First presentation of SPADE research on vowel variation and vowel space in English, from 400+ speakers, to the international sociolinguistics community. Great deal of interest in both the research, but also our new Integrated Speech Corpus ANalysis software, ISCAN. Led to offers of more datasets for the SPADE project.
Year(s) Of Engagement Activity 2018
URL https://wp.nyu.edu/nwav47/
 
Description workshop on Integrated Speech Corpus ANalysis (ISCAN) for SPADE, at NWAV47, New York University, 18 October 2018 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Hands-on workshop, with interactive tutorial, and presentation, on our new software system for SPADE, at NWAV47, New York University, 18 October 2018: Integrated Speech Corpus ANalysis - ISCAN: A new tool for large-scale, cross-corpus, sociolinguistic analysis: Jane Stuart-Smith, Morgan Sonderegger, Michael
McAuliffe . Well attended by sociolinguists working on English and other languages, including Arabic, Hebrew, French, Brazilian Portuguese and Occitan. Very enthusiastically received by all participants. Feedback during tutorial helped with ongoing development of software for the sociolinguistic community.
Year(s) Of Engagement Activity 2018
URL https://wp.nyu.edu/nwav47/workshops/