Statistical Natural Language Processing Methods for Computer Program Source Code

Complex software systems involve many components and make use of many external libraries. Programmers who work on such software must remember the protocols for using all of those components correctly, and the process of learning to use a new component can be time consuming and a source of bugs.

We believe that there is a major untapped resource that can help address this problem. Billions of lines of code are readily available on the Internet, much of which are of professional quality. Hidden within this code is a large amount of knowledge about good coding practices, for example, about avoiding error-prone constructs or about the best protocol for using a particular library. We envision a new type of programming tool, which could be called data-driven development tools, that aggregate knowledge about programming from a large corpus of mature software projects, for presentation within the development environment. Just as the current generation of IDEs helps developers to manage their code, the next generation of IDEs will help developers to learn how to write better code.

Fortunately, there is a research field that has already developed a large body of sophisticated tools for analyzing large amounts of text: namely, statistical natural language processing. The long-term strategic goal of this project is to develop new natural language processing techniques aimed at analyzing computer program source code, in order to help programmers learn coding techniques from the code of others. There is a large area for research here that has been almost completely unexplored.

As a first step in this research area, in this project we will focus on automatically identifying short code fragments, which we call idioms, that occur repeatedly across different software projects. An example of an idiom is the typical construct for iterating over an array in Java. Although they are ubiquitous in source code, idioms of this form have not to our knowledge been systematically studied, and we are unaware of any techniques for automatically identifying idioms. The main objective of this project is to develop new statistical NLP methods with the goal of automatically identifying idioms from a corpus of source code text. We call this research problem idiom mining, and it is to our knowledge a new research problem.

This is an interdisciplinary project that draws from statistical NLP, machine learning, and software engineering. The research work of this project is primarily in statistical NLP and machine learning, and will involve developing new statistical methods for finding idioms in programming language text.

The work in this proposal has the potential for substantial economic benefits in the long term. This project is an applied one with the general goal of building tools that help developers to create software more accurately and with less cost. The UK has one of the strongest software sectors in Europe. For example, in 2008 the UK accounted for 25% of European software companies. By making it possible to develop software at lower cost, we hope that this will benefit companies that sell software by lowering their costs. We hope that these tools would have special benefit to the many companies that develop custom software systems for their own in house use, by lowering the cost of these infrastructural projects.

We also hope that software developers themselves benefit, by having new tools that make their jobs easier and more enjoyable. It has been our personal experience as a professional software developer, that programmers find it extremely important to have good development tools, and even enjoy using them. This is evidenced by the fact that in many cases, programmers voluntarily spend their own time working on tools; development tools comprise many of the most successful open source projects, such as the Gnu command-line utilities, gcc, and Eclipse. We hope that development tools based on code mining have the potential to have an elusive "magic" quality, by finding patterns that programmers recognize but didn't realize existed---with the effect of making software developers happier and more productive.