Timeline-based-List-Question-Answering

Hi everyone! It is our NLP project in LLM, with authors: Box, MeowDai, Zhuoran, Barry. If you have any questions, you can contact Me (Box).

Project Introduction

We implemented in Timeline-based-List-Question-Answering problem, specifically the desire to be able to generate structured QA, by using a question and answer dataset containing time.

Project Structure

Our project contains multiple subtasks, you can click the different subtitles to check different Tasks.

KNN Few-shot

• KNN: K is top-k, and NN means Nearest Neighbour based on cosine similarity.
• Few-shot: The Q&A pairs found by KNN will fed to LLM model with original question.

Fine-tuning

• Fine-tuned based on the instructions tuning

RAG

• We complete the RAG on Wikipedia.
• The top-k entities are found through the retrieved context using similarity.
• We combined the retrieved text with the original question and fed it to LLM as a prompt.

RAG with time marker

• Similar to RAG, we introduce a Temporal Relevance metric and find temporal similarity.
• Combine the retrieved text, the time marker, and the original question as the prompt.

Evaluation Metrics

• Introduce Entity Match Metric, Timeline Match Metric, F1 Metric, Absolute Time Differences Metric, Completeness Metric