Preprint · arXiv · February 27, 2025
From Retrieval to Generation: Comparing Different Approaches
Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Ali, Mohammed, Adam Jatowt
Abstract
Knowledge-intensive tasks, particularly open-domain question answering (ODQA), document reranking, and retrieval-augmented language modeling, require a balance between retrieval accuracy and generative flexibility. Traditional retrieval models such as BM25 and Dense Passage Retrieval (DPR), efficiently retrieve from large corpora but often lack semantic depth. Generative models like GPT-4-o provide richer contextual understanding but face challenges in maintaining factual consistency. In this work, we conduct a systematic evaluation of retrieval-based, generation-based, and hybrid models, with a primary focus on their performance in ODQA and related retrieval-augmented tasks. Our results show that dense retrievers, particularly DPR, achieve strong performance in ODQA with a top-1 accuracy of 50.17\% on NQ, while hybrid models improve nDCG@10 scores on BEIR from 43.42 (BM25) to 52.59, demonstrating their strength in document reranking. Additionally, we analyze language modeling tasks using WikiText-103, showing that retrieval-based approaches like BM25 achieve lower perplexity compared to generative and hybrid methods, highlighting their utility in retrieval-augmented generation. By providing detailed comparisons and practical insights into the conditions where each approach excels, we aim to facilitate future optimizations in retrieval, reranking, and generative models for ODQA and related knowledge-intensive applications.
More by Abdelrahman Abdallah
- A Paired Printed-Handwritten Arabic Line Dataset for Handwriting Recognition and Cross-Modal RetrievalScientific Data · 2026
- A multilingual Arabic and English receipt dataset for text detection, recognition and information extractionZenodo · 2026
- UIBK-Embedding: Effective Text Retrieval by Unifying Dense and Late-Interaction RetrievalarXiv · 2026
- MemRevise: A Self-Revising Memory for Conversational SearcharXiv · 2026
