work / 07 / 96% accuracy · <800ms

Enterprise RAG at production scale

Hybrid dense + sparse retrieval with cross-encoder reranking across 50K+ enterprise documents. Top-5 retrieval precision lifted from 72% to 94%; latency cut from 4.2s to sub-800ms.

1. 01 Hybrid search: dense embeddings (OpenAI ada-002 plus BGE models fine-tuned on domain data) fused with sparse BM25 retrieval via Elasticsearch.
2. 02 Cross-encoder reranking (ms-marco-MiniLM) lifted top-5 retrieval precision from 72% to 94%.
3. 03 End-to-end latency cut from 4.2s to sub-800ms through Redis embedding caching, async chunk prefetching, and dynamic context-window packing.
4. 04 LLM evaluation harness holding 96% answer accuracy across 50K+ enterprise documents.