REFRAME: Rethinking the Essence, Flexibility and Reusability of Advanced Model Exploitation

Lead Research Organisation: University of Bristol
Department Name: Computer Science

Abstract

See the attached case for support.

Planned Impact

See the attached case for support.

Publications

10 25 50
 
Description Knowledge discovery aims to learn patterns from given data that are expected to apply to future data. Often there is relevant contextual information that, although it can have a considerable effect on the quality of the results, is rarely taken into account as it is not directly represented in the training data. The project has studied the role of context and context changes in the knowledge discovery process, and developed a range of practical techniques for context-aware machine learning and data mining, including changes in costs, data distribution and others. We have explored how data mining methodologies such as CRISP-DM can be extended to take context change into account.
Exploitation Route The project has resulted in a range of academic publications as well as software and tutorial materials for use of other researchers and practitioners.
Sectors Communities and Social Services/Policy,Creative Economy,Digital/Communication/Information Technologies (including Software),Healthcare

URL http://reframe-d2k.org/
 
Description Newton Call
Amount R$ 8,980 (BRL)
Organisation FACEPE 
Sector Public
Country Brazil
Start 06/2017 
End 12/2017
 
Description CHIST-ERA 
Organisation Institute of Genetics and Molecular and Cellular Biology (IGBMC)
Country France 
Sector Academic/University 
PI Contribution The REFRAME project was awarded under the CHIST-ERA scheme; each partner was funded by their national research council.
Collaborator Contribution All partners contributed to the REFRAME research programme.
Impact See http://reframe-d2k.org/Publications
Start Year 2013
 
Description CHIST-ERA 
Organisation Polytechnic University of Valencia
Country Spain 
Sector Academic/University 
PI Contribution The REFRAME project was awarded under the CHIST-ERA scheme; each partner was funded by their national research council.
Collaborator Contribution All partners contributed to the REFRAME research programme.
Impact See http://reframe-d2k.org/Publications
Start Year 2013
 
Title Precision-Recall-Gain curve software 
Description Precision-Recall analysis abounds in applications of binary classification where true negatives do not add value and hence should not affect assessment of the classifier's performance. Perhaps inspired by the many advantages of receiver operating characteristic (ROC) curves and the area under such curves for accuracy-based performance assessment, many researchers have taken to report Precision-Recall (PR) curves and associated areas as performance metric. We demonstrate in this work that this practice is fraught with difficulties, and show how to fix this by plotting PR curves in a different coordinate system. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2015 
Impact This work was selected for a plenary spotlight presentation at NIPS 2015. 
URL http://www.cs.bris.ac.uk/~flach/PRGcurves/
 
Title REFRAME Github repository 
Description This is a Github repository collecting the software outputs of the REFRAME project. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact The PRG software was supporting a publication that was selected for a spotlight presentation at NIPS'15. The tutorial repository was used in a tutorial presented at ECML-PKDD'16. 
URL https://github.com/REFRAME
 
Description LMCE 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact We organised two scientific workshops on Learning in Multiple Contexts at ECML-PKDD (LMCE) in the years 2014 (Nancy) and Porto (2015).
Year(s) Of Engagement Activity 2014,2015
URL http://users.dsic.upv.es/~flip/LMCE2015/
 
Description MoReBikeS challenge 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This was a data mining challenge organised in the context of the European Machine Learning and Data Mining conference (ECML-PKDD'15) in Porto.

The challenge is carried out in the framework of historical bicycle rental data obtained from Valencia, Spain. Bicycles are continuously taken from and returned to rental stations across the city. The data consists of time series describing hourly availability of bikes at each station; information on weather and (local) holidays is also provided. The challenge motivation is based on the fact that, while we may have had the opportunity to learn and tune good models for old stations with historical data, we do not always have the same amount of data for new stations. With that in mind, participants will receive, in addition to limited data for the new stations, a large number of trained models for old stations. The task will be to make predictions (3 hours ahead) with regard to the number of bikes available for these new stations and within the next months. This situation fluctuates considerably depending on the time of year, the station's location, etc. The key point here is that by using models from other stations that have been learnt from data spanning more than one year, better predictions can be made for the new stations. In the end, this challenge aims at promoting the reusability of models rather than retraining a different model again and again each time the context changes.
Year(s) Of Engagement Activity 2015
URL http://reframe-d2k.org/Challenge
 
Description Tutorial on Context-Aware Knowledge Discovery 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Traditionally, knowledge discovery aims to learn patterns from given data that are expected to apply to future data. Often there is relevant contextual information that, although it can have a considerable effect on the quality of the results, is rarely taken into account as it is not directly represented in the training data. This tutorial aims to elucidate the role of context and context changes in the knowledge discovery process, and to demonstrate how recent research advances in context-aware machine learning and data mining can be put to practical use. The tutorial will cover the main types of context changes, including changes in costs, data distribution and others. Participants will develop basic skills in choosing the appropriate modelling techniques and visualisation tools for the construction, selection, adaptation and understanding of versatile and context-aware models. We will discuss how data mining methodologies such as CRISP-DM can be extended to take context change into account. The tutorial will not only equip the attendees with new technical and methodological knowledge, but also encourage an anticipatory attitude towards context change.
Year(s) Of Engagement Activity 2016
URL https://github.com/REFRAME/tutorial