Making Big Data Exploration Work: Traceability and Usability

Big Data is something of a buzzword at present. It has many meanings, but typically is used to mean working with large, often massive, heterogeneous data sets, and in particular analysing and interpreting these data sets to extract information. Existing solutions typically emphasise parallel / distributed data management and analysis solutions, but pay insufficient attention to the user and usability. In this project, which will be driven by user needs, initially targeting the domains of oil and gas exploration and security, but also informed by engineering, geochemistry and bioinformatics, we will research and develop a series of tools to support the whole Big Data lifecycle.
The main lifecycle stages we will consider are:
i) efficient data acquisition, cleansing and semantic annotation
ii) visualisation of complex data: scalability and integration with expert knowledge
iii) scalable 3D immersive visualisation of complex data
iv) capturing user workflows in data processing and visualisation for reproducibility
Throughout the project focus will be placed on maintaining usability and developing components which are generic but capable of being customised to specific application domains so that domain experts will find the tools intuitive and accessible, reducing the barriers to Big Data analysis. A user panel will help to steer developments in the project, ensuring usability remains a clear focus.