ELIxIR QAS - Enhanced Large Language Model with Intelligent eXtraction and Information Retrieval based Question Answering System
Lead Research Organisation:
King's College London
Department Name: Informatics
Abstract
Proposed Research Questions as presented during 9-months Exam is as per below. However, feedback from 9-month Exam will now result in refinements to the RQ1. Also, RQ2, RQ3 will no longer be about Privacy, Bias impact from User Settings & Preferences in Question Answering System but instead linked to new RQ1
RQ#1 How can LLMs be used to improve performance of IR methods in handling diverse knowledge sources that integrate textual, tabular, and graph data structures?
RQ#2 How do user control mechanisms for privacy (user preferences, privacy settings for information sharing) impact IR methods in RQ1 ?
RQ#3 How do user privacy settings and privacy-bias trade-offs impact IR methods in RQ1 ?
Methodology (stated as per 9-month report , will be revised too)
Related Work : an extensive review LLM RAG based models would be undertaken to identify the state of the art , key challenges and gaps for Question Answering abilities of LLMs in Open as well as Closed Domain applications. The review is proposed to be focussed on LLM RAG based Architectural Choices for Knowledge Representation, Query Processing, Information Retrieval, Evaluation, User Privacy Preferences in dealing with Question Answering based on Textual and Knowledge Graph datasets as knowledge bases/sources for the Answers. It is proposed that such a review should be compared with traditional QAS and their strengths and deficiencies as a comparative for the study. State of the art architecture review of the available LLM+RAG models will be completed to identify the potential opportunities to use differential Information Retrieval Methods (for e.g. Vector Search for Textual Datasets and SPARQL query search for Knowledge Graphs) to adapt and improve existing LLM+RAG architectures through this research specifically in the context of Enterprise / Closed Domain QAS. Evaluation of such a system is proposed to be completed with Precision and Recall metrics as well as human (user of systems and domain experts) evaluation
RQ#1 How can LLMs be used to improve performance of IR methods in handling diverse knowledge sources that integrate textual, tabular, and graph data structures?
RQ#2 How do user control mechanisms for privacy (user preferences, privacy settings for information sharing) impact IR methods in RQ1 ?
RQ#3 How do user privacy settings and privacy-bias trade-offs impact IR methods in RQ1 ?
Methodology (stated as per 9-month report , will be revised too)
Related Work : an extensive review LLM RAG based models would be undertaken to identify the state of the art , key challenges and gaps for Question Answering abilities of LLMs in Open as well as Closed Domain applications. The review is proposed to be focussed on LLM RAG based Architectural Choices for Knowledge Representation, Query Processing, Information Retrieval, Evaluation, User Privacy Preferences in dealing with Question Answering based on Textual and Knowledge Graph datasets as knowledge bases/sources for the Answers. It is proposed that such a review should be compared with traditional QAS and their strengths and deficiencies as a comparative for the study. State of the art architecture review of the available LLM+RAG models will be completed to identify the potential opportunities to use differential Information Retrieval Methods (for e.g. Vector Search for Textual Datasets and SPARQL query search for Knowledge Graphs) to adapt and improve existing LLM+RAG architectures through this research specifically in the context of Enterprise / Closed Domain QAS. Evaluation of such a system is proposed to be completed with Precision and Recall metrics as well as human (user of systems and domain experts) evaluation
People |
ORCID iD |
| Arunav Das (Student) |
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/Y528572/1 | 30/09/2023 | 29/09/2028 | |||
| 2888808 | Studentship | EP/Y528572/1 | 30/09/2023 | 29/09/2027 | Arunav Das |