Reranking

Reranking is a two-stage retrieval process where an initial fast retrieval step (vector similarity or BM25) fetches candidate documents, and a second, more computationally expensive model reorders them by relevance to the query. This produces higher-quality context for LLMs in RAG pipelines and improves search precision. ¹⁾

Initial retrievers compress documents into fixed-size vector embeddings, which limits their ability to capture nuanced semantic relationships between queries and documents. Rerankers use cross-attention mechanisms to jointly process query-document pairs, enabling deeper alignment assessment. This typically yields 20-50% gains in metrics like nDCG@10, MRR, or Hit Rate. ^{2) 3)}

Bi-encoders (used in initial retrieval) encode queries and documents separately into independent embeddings. This enables fast approximate nearest neighbor search across millions of documents but limits the interaction between query and document representations. ⁴⁾

Cross-encoders (used in reranking) jointly process query-document pairs with full cross-attention, capturing fine-grained relevance at higher computational cost. They are applied only to small candidate sets (typically top-100 to top-1000).

Cohere Rerank v3 and v3.5 are cross-encoder models optimized for RAG applications. V3.5 improves multilingual support and reduces latency. They integrate easily into existing retrieval pipelines and are commonly used with Cohere embeddings for two-stage retrieval. ⁵⁾

Open-source cross-encoder from the Beijing Academy of Artificial Intelligence. Excels in dense retrieval reranking with high accuracy on standard benchmarks, making it a strong choice for enterprise RAG systems that require self-hosting. ⁶⁾

Late interaction models that tokenize queries and documents into separate embeddings and compute fine-grained token-level similarities without full cross-attention. ColBERTv2 optimizes efficiency through compression. ⁷⁾

• Balances speed and precision
• Ideal for large-scale reranking where cross-encoder latency is prohibitive

Production-focused cross-encoder for multilingual tasks with lightweight inference and strong semantic matching. Commonly used in RAG pipelines requiring domain adaptation. ⁸⁾

Fast, local cross-encoder optimized with ONNX for low-latency inference without API calls. Suited for real-time systems where external API latency is unacceptable. ⁹⁾

Prompts LLMs (e.g., GPT-4) for pairwise or listwise scoring of candidate documents. Leverages LLM reasoning capabilities and adapts to new domains via few-shot learning, but incurs high cost and latency. ^{10) 11)}

• Reranking typically yields 20-50% gains in nDCG@10, MRR, or Hit Rate
• Cross-encoders like Cohere and BGE outperform bi-encoders by 10-30% on semantic retrieval tasks
• ColBERT approaches cross-encoder quality at lower latency
• LLM-based rerankers (RankGPT) excel on complex queries but lag in speed ¹²⁾

Rerankers add 2-10x latency compared to initial retrieval (typically 50-200ms for top-100 candidates on GPU). Latency scales with the number of candidates processed. ¹³⁾

• Score-based methods (RRF) are fastest
• Neural cross-encoders are moderate
• LLM-based rerankers are slowest

Use distillation, quantization, or FlashRank for latency-sensitive applications.

Use when:

• Precision is critical (enterprise search, customer support, specialized RAG)
• Latency and cost are tolerable
• Initial recall is good (top-k > 50 candidates available)

Skip when:

• High-traffic, low-latency requirements (real-time chat at massive scale)
• Simple keyword search suffices for the use case
• Corpus is very small (under a few hundred documents) ¹⁴⁾

The standard retrieve-then-rerank pattern:

1. Retrieve top 100-1,000 candidates via bi-encoder or hybrid search (fast first stage)
2. Rerank to top 5-20 documents using a cross-encoder (precise second stage)
3. Pass reranked chunks to the LLM generator for response synthesis

This pattern maximizes efficiency by using the fast first stage to narrow scope for the expensive reranking step. ¹⁵⁾

• Retrieval Strategies
• Hybrid Search
• Semantic Search
• Embedding Models Comparison