uComp: Embedded Human Computation for Knowledge Extraction and Evaluation

The rapid growth and fragmented character of social media and publicly available structured data challenges established approaches to knowledge extraction. Many algorithms fail when they encounter noisy, multilingual and often contradictory input. Efforts to increase the reliability and scalability of these algorithms face a lack of suitable training data and gold standards. Given that humans excel at interpreting contradictory and context-dependent language data, the uComp project will address the above mentioned shortcomings by merging collective human intelligence and automated methods in a symbiotic fashion. The project will build upon the emerging field of Human Computation (HC) in the tradition of games with a purpose and crowdsourcing marketplaces. It will advance the field of Web Science by developing a scalable and generic HC framework for knowledge extraction and evaluation, delegating the most challenging tasks to large communities of users and continuously learning from their input to optimise automated methods as part of an iterative process. A major contribution is the proposed foundational research on Embedded Human Computation (EHC), which will advance and integrate the currently fragmented research on human and machine computation. EHC goes beyond mere data collection and embeds the HC paradigm into adaptive knowledge extraction workflows. An open evaluation campaign will validate the accuracy and scalability of EHC to acquire factual and affective knowledge. In addition to novel evaluation methods, uComp will also provide shared datasets and benchmark the EHC approach against established knowledge processing algorithms.

While the methods of uComp will be held generic to be evaluated across domains, climate change was chosen as the main use case for its challenging nature, subject to changing and conflicting interpretations. Active collaboration with international organisations (EEA, NOAA, NASA) will increase the project's visibility and promote the adoption of the EHC paradigm among a wide range of stakeholders.

The uComp project aims to harness the power of Human Computation (HC) for scaling up and improving the accuracy of deep knowledge acquisition from noisy, multilingual data. It spans two novel ICT disciplines: Human Computation and Web Science. The latter is one of CHIST-ERA's target research areas. The 2010 FET Consultation also identifies Web Science as a key emerging discipline, which requires breakthrough foundational research and where academic impact could be significant. As stated in the report, harnessing collective intelligence comes with "large and novel problems as to have genuinely revolutionary potential". uComp aims to tackle these new challenges, in order to deliver the necessary fundamental understanding and computational methods, and to ultimately enable innovative HC-based digital applications. The project will thus directly contribute to the target impacts of FET Objective ICT-2011.9.1: Challenging current Thinking, through an "ambitious proof-of-concept and its supporting scientific foundation, where novelty comes from new, high-risk ideas rather than from the refinement of current ICT approaches".

Another major impact area comes from uComp's application domain, i.e., climate change, which was chosen for its challenging nature in line with the CHIST-ERA call, and the match with the JPI Climate strategic research agenda. Joint Programming Initiatives (JPI) implement the European Research Area (ERA). Twelve member states have taken up this approach and promote a new JPI on "Connecting Climate Knowledge for Europe". JPI Climate identifies climate change as a complex reality that affects European society at large, and calls for knowledge-based information and services to respond to stakeholder needs. The active collaboration with the European Environment Agency (EEA), the British Library, the National Oceanic and Atmospheric Administration (NOAA) and the NASA Ames Research Center will ensure a high international visibility and societal impact, and the widespread adoption of EHC methods among participating scientists. Thereby uComp will support stakeholder collaboration and engage citizens in scientific research (citizen science).

On UK level, the British Library has a Memorandum of Understanding with the Living With Environmental Change (LWEC) programme, a partnership comprising the 22 major public sector funders and producers of environmental information, to work together to improve access and discovery of environmental information. Through this relationship, results from this work will have a significant, wide impact and may be applied by a wide range of stakeholders well beyond those involved in this project.

The project will also carry out detailed investigation of the commercial exploitation potential of uComp as an enabler of active business ecosystems around the EHC concept. uComp will devise strategies to attract research and commercial partners by aligning the scientific progress with those partners' specific needs. We will also undertake a continuous 'market watch', to analyse and explore different business opportunities, which will be turned into particular action in the exploitation plan, viable for prospective commercialisation of some of the results after the end of the project. Activities towards standardisation of the project results and collaboration with other projects and relevant initiatives will also be explored and coordinated, so that uComp can have the best possible impact both in the scientific and commercial communities.