Foundational Structures for Compositional Meaning

Words are the building blocks of sentences, yet meaning of a sentence goes well beyond meanings of the words therein. Indeed, while we do have dictionaries for words, we don't seem to need them to infer the meaning of a sentence from meanings of its constituents. Discovering the process of meaning assignment in natural languages is one of the most foundational issues in linguistics and computer science, whose findings will increase our understanding of cognition and intelligence and may assist in applications to automating language-related tasks, such as document search as done by Google.

To date, the compositional logical and the distributional probabilistic models have provided two complementary partial solutions to the problem of meaning assigning in natural languages. The logical approach is based on classic ideas from mathematical logic, mainly Frege's principle that meaning of a sentence can be derived from the relations of the words in it. The distributional model is more recent, it can be related to Wittgenstein's philosophy of `meaning as use', whereby meanings of words can be determined from their context. The logical models have been the champions on the theory side, whereas in practice their probabilistic rivals have provided the best predictions. This two-sortedness of defining properties of meaning: `logical form' versus `contextual use', has left the question of `what is the foundational structure of meaning?' even more open a question than before. This project has ambitious and far-reaching goals; it aims to bring together these two complementary concepts to tackle the question. And it aims to do so by bridging the fields of linguistics, computer science, logic, probability theory, category theory, and even physics. Its scope is foundational, multi and inter disciplinary, with an eye towards applications.

Meaning assignment is a dynamic interactive process involving grammar and logic as well as meanings of words. Both of the two existing approaches to language miss a crucial aspect of this process: the logical model ignores meanings of words, the distributional model ignores the grammar and logic. We aim to model the entire dynamic process alongside the following three strands of integration, foundations, and applications.

(I) In integration we develop a process of meaning assignment that acts with the compositional forms of the logical model on the contextual word-meaning entities of the distributional model.

(II) In foundations, we go beyond classical logical principles of compositionality and context-based models of meaning to develop more fundamental processes of meaning assignments based on novel information-flow techniques, mainly from physics, but also from other linguistic approaches and other models of word meaning, such as ontological domains and conceptual spaces.

(III) In applications, we evaluate our theories against naturally occurring data and apply the results to practical issues based on meaning inference and similarity, e.g. in search. To be able to work with logical connectives in Google, one needs to re-enter them by hand in the `advanced search' tab, by manually decomposing the logical structure of the sentence and moreover providing the extra context for their different meanings. This is fundamentally non-compositional and goes against the spirit of automated search. It is exactly here that the lack of compositional methods in meaning assignment causes practical problems and where our compositional methods become of use. Hence, we aim to put forward our results to tackle such problems, e.g. to be able to use our sentence similarity models for paraphrasing, question-answering, and retrieving documents that have the same meaning and/or are about the same subject. Our proposed partnership with Google, ensures access to real life data and helps implementation and applicability of our methods in small and large scales.

On the knowledge side, the proposed research will cause significant scientific advances across different disciplines of logic, linguistics, mathematics, physics, and computer science. This is by modeling the process of cognition and natural language generation and developing new mathematics, logic, and high level diagrammatic tools.

The project has 3 partners, from Computer Science in Cambridge, Cognitive Sciences and Artificial Intelligence in Utrecht, and industry in Google. These extend the geographical boundaries of the impacts of the project from UK to Europe and the US, but also from academia to industry. I also have on-going collaborations with experts from these various disciplines in venues including UK, Italy, France, USA, and Canada.

On the economy and social side, internet with its huge pool of services and data has become an inseparable part of our daily lives. The theoretical results of this project will be put forward to improve the quality of services on the internet. At the moment documents are identifies as bags of their words. If the relationships between the words is also taken into account, language processing tasks will hugely benefit, for instance tasks such as information retrieval from text and document search. As a result, new techniques for applications such as question answering and textual entailment will be developed, better answers to online questions will be provided, and more comprehensible summaries of news and articles will be constructed automatically. The partnership with Google and Cambridge is exactly towards following and realizing this pathway.

From the other side of the spectrum, the results will help us understand the nature of intelligence and language understanding. This has conceptual importance of its own, will improve the quality of human life in due time, by facilitating mutual understanding in society and across societies of different languages.

Finally, the proposed theoretical setting is based on using simple diagrammatic techniques to depict mathematical structure and logic. The simplicity and accessibility of these methods provides the public with a chance to understand advanced academic developments, a chance which will have an impact on educating the society. We have had open sessions to introduce Computer Science research to high school students and especially to girls in Oxford. The diagrammatic methods and their application areas caused much discussion with and within the students.

On the academic side, apart from publishing articles and attending already-established workshops and conferences, I have asked for funding to organize two workshops. This is to fill the interdisciplinary gap and bring together researchers from the different disciplines of the project, so that we can discuss and disseminate ideas and results and help start a multi-subject community across these different fields. I will also organize the interdisciplinary seminar series of the logic group of computing lab at oxford. These are open to all academic fields and also the public. Other impacts are through training and teaching. I have asked funding for a doctorate student and plan to continue lecturing my field of expertise based on the project. I have already been invited to give advanced lecture series about the subject in Utrecht and in a Masters course in Cognitive Science in University of Latvia.