Generative Kernels and Score Spaces for Classification of Speech

The aim of this project is to significantly improve the performance of automatic speech recognition systems across a wide-range of environments, speakers and speaking styles. The performance of state-of-the-art speech recognition systems is often acceptable under fairly controlled conditions and where the levels of background noise are low. However for many realistic situations there can be high levels of background noise, for example in-car navigation, or widely ranging channel conditions and speaking styles, such as observed on YouTube-style data. This fragility of speech recognition systems is one of the primary reasons that speech recognition systems are not more widely deployed and used. It limits the possible domains in which speech can be reliably used, and increases the cost of developing applications as systems must be tuned to limit the impact of this fragility. This includes collecting domain specific data and significant tuning of the application itself.The vast majority of research for speech recognition has concentrated on improving the performance of hidden Markov model (HMM) based systems. HMMs are an example of a generative model and are currently used in state-of-the-art speech recognition systems. A wide number of approaches have been developed to improve the performance of these systems under speaker and noise changes. Despite these approaches, systems are not sufficiently robust to allow speech recognition systems to achieve the level of impact that the naturalness of the interface should allow. This project will combine the current generative models developed in the speech community with discriminative classifiers used in both the speech and machine learning communities. An important, novel, aspect of the proposed approach is that the generative models are used to define a score-space that can be used as features by the discriminative classifiers. This approach has a number of advantages. It is possible to use current state-of-the-art adaptation and robustness approaches to compensate the acoustic models for particular speakers and noise conditions. As well as enabling any advances in these approaches to be incorporated into the scheme, it is not necessary to develop approaches that adapt the discriminative classifiers to speakers, style and noise. One of the major problems with speech recognition is that variable length data sequences must be classified. Using generative models also allows the dynamic aspects of speech data to be handled without having to alter the discriminative classifier. The final advantage is the nature of the score-space obtained from the generative model. Generative models such as HMMs have underlying conditional independence assumptions that, whilst enabling them to efficiently represent data sequences, do not accurately represent the dependencies in data sequences such as speech. The score-space associated with a generative model does not have the same conditional independence assumptions as the original generative model. This allows more accurate modelling of the dependencies in the speech data.The combination of generative and discriminative classifiers will be investigated on two very difficult forms of data that current systems perform badly on. The first task is adverse environment recognition of speech. In these situations there are very high levels of background noise which causes severe degradation in system performance. Data of interest for this task will be specified in collaboration with Toshiba Research Europe Ltd. The second task of interest is large vocabulary speech recognition of data from a wide-range of speaking styles and conditions. Google has supplied transcribed data from YouTube to allow evaluation of systems on highly diverse data. The project will yield significant performance gains over current state-of-the-art approaches for both tasks.

The growth of business based on speech-enabled technology has been slower than predicted. A major contributing factor to this slow growth is that speech recognition systems are still not sufficiently robust to changing background noise conditions, speaker-styles, and accents. This results in unacceptable performance for too many users. Furthermore the cost of development of these applications is large as data is typically collected for the specific target domain and the application tuned to reduce the impact of the current fragility of speech recognition systems. Any approach that yields significant improvements in robustness (to noise, speaker and domain changes) would therefore be of enormous direct benefit to the speech industry making many new applications of the technology feasible. A range of companies in the UK would benefit in this case from core speech technology providers, such as Autonomy and Toshiba Research Europe Ltd, to application providers, such as Telephonetics, Acuvoice, through to application designers, such as VoxGen and Edify. The companies, such as major airlines, banks and new media, who would like to make further use of speech recognition to reduce operating costs and enable new applications would also benefit. The outcome of this research will be shared in the first instance with providers of core speech recognition technology. The Speech Group at CUED has close collaborations with a number of UK and international speech companies including Toshiba Research Europe Ltd (TREL), Google and IBM. Data from existing collaborations with TREL and Google will be used to benchmark the technology created within this research project. This will allow the companies to easily identify technical advances over their existing technology. In addition to research publications including conference papers and technical reports, software implementations of the research outcomes will be made available. The software will be released as an extension to the existing HTK toolkit via the HTK website. This will enable broader industry to replicate the results on publicly available databases.