====== Reranking ======

Reranking is a two-stage retrieval process where an initial fast retrieval step (vector similarity or BM25) fetches candidate documents, and a second, more computationally expensive model reorders them by relevance to the query. This produces higher-quality context for LLMs in RAG pipelines and improves search precision. ((https://www.pinecone.io/learn/series/rag/rerankers/|Pinecone: Rerankers for RAG))

===== Why Reranking Is Needed =====

Initial retrievers compress documents into fixed-size vector embeddings, which limits their ability to capture nuanced semantic relationships between queries and documents. Rerankers use cross-attention mechanisms to jointly process query-document pairs, enabling deeper alignment assessment. This typically yields 20-50% gains in metrics like nDCG@10, MRR, or Hit Rate. ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking)) ((https://zilliz.com/learn/optimize-rag-with-rerankers-the-role-and-tradeoffs|Zilliz: Optimize RAG with Rerankers))

===== Cross-Encoders vs Bi-Encoders =====

**Bi-encoders** (used in initial retrieval) encode queries and documents separately into independent embeddings. This enables fast approximate nearest neighbor search across millions of documents but limits the interaction between query and document representations. ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

**Cross-encoders** (used in reranking) jointly process query-document pairs with full cross-attention, capturing fine-grained relevance at higher computational cost. They are applied only to small candidate sets (typically top-100 to top-1000).

===== Reranker Models =====

==== Cohere Rerank ====

Cohere Rerank v3 and v3.5 are cross-encoder models optimized for RAG applications. V3.5 improves multilingual support and reduces latency. They integrate easily into existing retrieval pipelines and are commonly used with Cohere embeddings for two-stage retrieval. ((https://aws.amazon.com/blogs/machine-learning/improve-rag-performance-using-cohere-rerank/|AWS: Cohere Rerank for RAG))

==== BGE Reranker (BAAI) ====

Open-source cross-encoder from the Beijing Academy of Artificial Intelligence. Excels in dense retrieval reranking with high accuracy on standard benchmarks, making it a strong choice for enterprise RAG systems that require self-hosting. ((https://zilliz.com/learn/optimize-rag-with-rerankers-the-role-and-tradeoffs|Zilliz: Rerankers))

==== ColBERT and ColBERTv2 ====

Late interaction models that tokenize queries and documents into separate embeddings and compute fine-grained token-level similarities without full cross-attention. ColBERTv2 optimizes efficiency through compression. ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

  * Balances speed and precision
  * Ideal for large-scale reranking where cross-encoder latency is prohibitive

==== Jina Reranker ====

Production-focused cross-encoder for multilingual tasks with lightweight inference and strong semantic matching. Commonly used in RAG pipelines requiring domain adaptation. ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

==== FlashRank ====

Fast, local cross-encoder optimized with ONNX for low-latency inference without API calls. Suited for real-time systems where external API latency is unacceptable. ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

==== RankGPT (LLM-Based Reranking) ====

Prompts LLMs (e.g., GPT-4) for pairwise or listwise scoring of candidate documents. Leverages LLM reasoning capabilities and adapts to new domains via few-shot learning, but incurs high cost and latency. ((https://developer.nvidia.com/blog/enhancing-rag-pipelines-with-re-ranking/|NVIDIA: Enhancing RAG with Re-Ranking)) ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

===== Performance =====

  * Reranking typically yields **20-50% gains** in nDCG@10, MRR, or Hit Rate
  * Cross-encoders like Cohere and BGE outperform bi-encoders by 10-30% on semantic retrieval tasks
  * ColBERT approaches cross-encoder quality at lower latency
  * LLM-based rerankers (RankGPT) excel on complex queries but lag in speed ((https://www.chatbase.co/blog/reranking|Chatbase: Reranking))

===== Latency Considerations =====

Rerankers add 2-10x latency compared to initial retrieval (typically 50-200ms for top-100 candidates on GPU). Latency scales with the number of candidates processed. ((https://zilliz.com/learn/optimize-rag-with-rerankers-the-role-and-tradeoffs|Zilliz: Rerankers))

  * Score-based methods (RRF) are fastest
  * Neural cross-encoders are moderate
  * LLM-based rerankers are slowest

Use distillation, quantization, or FlashRank for latency-sensitive applications.

===== When to Use Reranking =====

**Use when:**
  * Precision is critical (enterprise search, customer support, specialized RAG)
  * Latency and cost are tolerable
  * Initial recall is good (top-k > 50 candidates available)

**Skip when:**
  * High-traffic, low-latency requirements (real-time chat at massive scale)
  * Simple keyword search suffices for the use case
  * Corpus is very small (under a few hundred documents) ((https://zilliz.com/learn/optimize-rag-with-rerankers-the-role-and-tradeoffs|Zilliz: Rerankers))

===== Implementation Pattern =====

The standard retrieve-then-rerank pattern:

  - **Retrieve** top 100-1,000 candidates via bi-encoder or hybrid search (fast first stage)
  - **Rerank** to top 5-20 documents using a cross-encoder (precise second stage)
  - **Pass** reranked chunks to the LLM generator for response synthesis

This pattern maximizes efficiency by using the fast first stage to narrow scope for the expensive reranking step. ((https://www.pinecone.io/learn/series/rag/rerankers/|Pinecone: Rerankers))

===== See Also =====

  * [[retrieval_strategies|Retrieval Strategies]]
  * [[hybrid_search|Hybrid Search]]
  * [[semantic_search|Semantic Search]]
  * [[embedding_models_comparison|Embedding Models Comparison]]

===== References =====