Matheuristics for multi-criterion data clustering: towards multi-criterion big data analytics

With rapid increases in data volume in all areas of life, the meaningful analysis of these data is becoming a crucial bottleneck. Whether data are generated by customer transactions, through communications on social media, or as a by-product of manufacturing processes, data are meaningless unless suitable techniques are available to select the most relevant data, analyze these data and turn raw data into tangible information and insight. To some extent, "big data" reverses traditional approaches in data-mining, as data collection now frequently precedes the definition of an actual question or hypothesis. The purported advantage of this approach is that novel, unexpected findings may materialize - a premise that relies, however, on the expert use of suitable approaches for exploratory data analysis. The prominence of "big data" therefore fuels the need and use of scalable and powerful approaches to exploratory data analysis. Data clustering techniques present one of the most fundamental tools in exploratory data analysis, and this project aims to deliver novel techniques that are accurate, flexible and scalable to large data sets.

Data clustering techniques present one of the most fundamental tools in exploratory data analysis. Conceptually, data clustering refers to the identification of sub-groups within a data set so that items within the same group are similar and those in different groups are dissimilar; e.g., in the context of insurance data, a "cluster" of people may relate to customers who show similar behaviour in their claim patterns over time, while those in different clusters behave differently. Mathematically, data clustering can be seen as an example of a problem where good solutions are best described using a set of different criteria that account for conflicting properties such as the compactness of clusters and the separation between clusters.

The above observation has recently led to the development of multi-criterion approaches to data clustering, which explicitly consider a number of clustering criteria. This approach has shown a lot of promise, in terms of the accuracy and the robustness of the solutions obtained. However, current techniques for multi-criterion clustering are limited regarding their scalability to very large data sets and also their flexibility with respect to their consideration of different sources of dissimilarity data. This project proposes a novel technique for multi-criterion clustering: the algorithm will combine complementary ideas from two sub-fields of computer science, leading to improved scalability and flexibility of the technique developed. The work will include the development of an interactive user-interface and the application of multi-criterion clustering to problems in finance and marketing. All software produced will be released publicly.

Briefly, the non-academic beneficiaries of this project are:
- Users of exploratory data analysis in industry / commerce, including manufacturing as a major source of "big data".
- Commercial providers of software tools for data-mining.
- The wider public, as they benefit from improved data analytical techniques through the spin-offs of such knowledge, e.g. through the development of improved recommendation systems (marketing), the identification of groups of co-regulated genes (biology), etc.

One of the applications considered in this proposal (work package WP4.2) involves measures of customer experience and loyalty and will likely involve industrial partners as the primary source of these data. Further opportunities for collaborations will be sought, e.g. as a part of industrial dissertation projects. The establishment of fruitful collaborations within and outside academia is supported by the third objective of the proposal, which aims to embed multi-criterion clustering techniques into an interactive, easy-to-use interface that is suitable for the rapid training and use by students, fellow academics, industrial partners etc. Similarly, work packages WP4.1 and WP4.2 support exploitation activities directly, as they deliver practical application examples that can facilitate and underpin the dissemination of the approach.