Unsupervised Background Knowledge for Language Understanding

Significant progress in Artificial Intelligence (AI) has been made in recent years, and this has resulted in a huge expectation on what this technology can offer us in the future. However, there are still many challenges that must be addressed before this promise can be turned into a reality, and one of these challenges is Natural Language Processing (NLP). If a computer is ever to understand humans in a natural way and to demonstrate a level of intelligence that we would normally expect, then the problem of Language Understanding must be solved.

Making computers to understand natural languages is a non-trivial task. Current approaches to language understanding rely on end-to-end supervised learning, exemplified by deep learning techniques in recent years. Typically, a corpus of relevant text is collected and then used to train the computer to perform a certain task. However, this approach may have several problems, e.g., the words extracted and used to train a computer often have implicit meanings and can be ambiguous. Consider the following two sentences, for example:

(1) We found many birds during our visit to the zoo: eagles, parrots, cranes...
(2) The crane was hurt and could barely move.

A computer will not be able to understand from these training examples that there are in fact two types of crane (bird and machine) and the fact that only one type of crane (bird) can get hurt. It is widely recognised that handling word ambiguity and, more broadly, understanding what words mean, is a significant challenge in NLP. For instance, Google Translate, widely considered as the state-of-the-art in machine translation, fails to translate these two sentences correctly even to closely related languages such as Spanish. Generally speaking, current techniques are hard to generalize across different tasks and domains, especially in applications requiring language understanding.

The proposed research intends to develop theories and novel solutions to bridge this gap by combining and leveraging lexical resources and unsupervised techniques for analysing text corpora, thereby learning the much-needed, but not-explicitly-available background knowledge. Our goal is then to seamlessly integrate this background knowledge into real-world applications for more accurate language understanding. We will exploit these techniques in different languages, making them directly applicable in important multilingual NLP tasks, including lower-resourced languages such as Welsh, and in domains with direct societal impact such as social media and health care.

The beneficiaries of this fellowship are diverse. Natural Language Processing (NLP) and text mining practitioners and researchers will directly benefit from the resources and models that we will develop. Moreover, researchers in other fields (e.g. social scientists) can benefit from our tools to understand textual content in different languages. In the longer term, better language understanding systems will impact end-users in their daily interactions: better virtual assistants, more accurate translation systems, etc.

Industry will benefit from the fact that we will train three postdoctoral research associates, including training and development in other top research labs, which will provide them with complementary expertise. Moreover, companies will also benefit from the large-scale lexical resource that we will make available and the fact that we will release open-source practical NLP models that can be used for different domains and languages.

As far as our first case study is concerned, this fellowship will provide tools to analyze and fight disinformation campaigns on social media. This will benefit the general public, as disinformation is currently an important problem threatening important aspects of our society, including democracy. In general, detecting deceptive content can help public security and it can provide a useful tool for addressing this issue directly from police and government bodies, including policymakers. Our second case study will provide better insights to develop treatments for patients with mental health. In addition to saving money from NHS, it can potentially improve the general wellbeing of patients with psychiatric disorders. Moreover, it will open up avenues for future research, where automatic tools will become an important tool to analyse available unstructured data in the form of e.g. clinical records.