Justified Assessments of Service Provider Reputation

Lead Research Organisation: University of Warwick
Department Name: Computer Science

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description Bootstrapping Mechanism: In highly dynamic marketplaces (where members join and leave with high frequencies), information about specific members is sparse. Stereotypes, such as "colourful logo" can be used to improve the accuracy of trust assessments when relevant information is not available, but communicating stereotype observations to other members is problematic when members have different viewpoints or perspectives. Asking for a recommendation about a "colourful logo", may be meaningless to someone who is colour-blind. We proposed a method for accommodating such misunderstandings by using a translation function, which we show to make stereotype recommendations useful even where agents have different viewpoints.

Incentivisation Framework: We have developed an incentivisation framework, incorporating various incentives that will encourage service providers to release reports of the circumstances under which their interactions with clients took place. Knowledge of such circumstances gives individuals useful information to support their decision-making in selecting a future interaction partner. Provenance data provides a suitable solution for exposing information on various circumstances underlying a service provision. Providers are the obvious source of such provenance data, as it is a record of how they provided a service, but they may not be willing to release such records for several reasons. This may be due to the additional burden incurred on the provider side (the process of provenance recording and documentation could be tedious and expensive), or for competitive grounds (e.g. it may be against provider interests to release records showing that they performed poorly). Providing relevant incentives to providers is a promising way to encourage them to release provenance data. For example, in the context of service-oriented marketplaces, reputation is a particularly attractive (extrinsic) incentive for service providers, as reputation is commonly utilised by clients to assess the degree of risk associated with providers.

Delegation Hierarchy Interrogator: A provider accomplishing a complex task (composite service) may delegate sub-tasks to sub-providers, which in turn may sub-contract some sub-tasks to others, resulting in a multi-level delegation hierarchy. Assessing the reputation of the composite service provider without interrogating this delegation hierarchy might yield an inaccurate and an unfair assessment for the provider. For example, a low reputation of provider due to the past failures of its transportation sub-provider might give an unfair view if the sub-provider has been changed to avoid such failures re-occurring. We therefore developed a delegation hierarchy interrogator that accounts for the various levels of sub-provision, and for different types of dependency among sub-providers, in order to allow a more accurate and fair assessment of composite service providers.
Exploitation Route Both the Bootstrapping Mechanism and the Delegation Hierarchy Interrogator will advance the field of automated reputation assessments. The former will improve accuracy of assessments where limited direct experience with a provider is available. The latter provides more accurate and fair assessments of providers that utilise sub-delegation, allowing their success or failure to be assessed according to responsibility.

The Incentivisation Framework would contribute towards altering the behaviour of service providers with respect to provenance provision. This in turn would improve automated decision making, since the availability of provenance information would enable individuals to make more informed decisions (e.g. provider selection decisions).
Sectors Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Manufacturing, including Industrial Biotechology,Transport
URL https://sites.google.com/site/jasprepsrc/publications
 
Description Impact Acceleration Award
Amount £7,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 05/2016 
End 09/2016
 
Title Machine Learning Reputation Test Bed 
Description The Machine Learning Reputation Test Bed (MLRTB) has been developed as part of JASPR for research into trust and reputation in agent based systems. The aim of MLRTB is to enable development of reputation models driven by provenance records, generated by interactions and stored in a provenance store. Reputation assessment models process these records and produce reputation scores of agents in the simulation, which are then used to inform decisions about future interactions. Our goal is to use machine learning, provided by the WEKA and MEKA machine learning toolkits, in learning these provenance records and predicting future interaction outcomes. Developing new reputation assessment models in MLRTB is straightforward, and implementations of several existing reputation assessment models are provided for comparison, including FIRE, BRS, TRAVOS, STAGE, BLADE, and HABIT. Written in Scala and Java, the MLRTB is extensible to new environments with different kinds of interaction and reputation assessments. There are currently three marketplace environments implemented that each focus on different aspects of reputation assessment. A logistics simulation is focussed on reputation of provider agents in compositions, a bootstrapping simulation introduces provider stereotypes, and a marketplace simulation builds on this by also modelling user and service contexts. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact This software has enabled research into reputation context and stereotypes. Specifically, it has been used to evaluate our reputation assessment model in our extended abstract paper published at AAMAS 2017, "Bootstrapping trust with partial and subjective observability", as well as several other works in progress. 
URL https://github.com/jaspr-project/MLReputationTestBed
 
Description Digital Catapult Centre workshop on Online Reputation (London) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact We gave a talk at the workshop on the research ideas of JASPR, and our impact and engagement activities in this project. Specifically, we discussed the limitations of existing ways to recommend services based on reputation, and how JASPR's ideas of taking account of the context in which services are used could improve personalised assessment and recommendation. We also discussed industry case studies being explored as part of JASPR's impact plan. Our primary purpose of giving the talk, which was invited, was to expose our ideas to wider industry, to aid future exploitation or obtaining further research or innovation funding.
Year(s) Of Engagement Activity 2016
URL https://www.digitalcatapultcentre.org.uk/event/online-reputation-and-opinion-formation/
 
Description School of Electronics and Computer Science, University of Southampton 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact We gave a talk on the JASPR research ideas to an academic department at the University of Southampton, including academics and postgraduate students.
Year(s) Of Engagement Activity 2016
URL http://www.ecs.soton.ac.uk/