RAG Platform
Generative AI Platform
An enterprise Generative AI platform built to answer questions over a corpus of 10M+ documents across multiple customer domains, each mapped to its own embedding store with RBAC-based access control. The system combines retrieval-augmented generation with a multi-agent framework so that every query is routed to the agent best equipped to answer it.
Key Features:
- Hybrid RAG Architecture: Hierarchical retrieval with document-level and chunk-level indexes, hybrid search (70% vector + 30% keyword) on Azure AI Search, and cross-encoder reranking using ms-marco-MiniLM.
- Multi-Agent Orchestration: A coordinator agent built with LangGraph routes requests to specialized Legal, Financial, and Support agents, each connected to domain-specific indexes with conditional routing and confidence-based verification.
- Semantic Caching: Redis-backed semantic cache at a 0.95 similarity threshold, delivering a 3x latency improvement and 40% LLM cost reduction.
- Data Ingestion Pipeline: Azure Data Factory and Azure Functions process 100K+ documents daily with classification, OCR (Tesseract), intelligent chunking, dead-letter queues, and checkpointing.
- LLMOps: Prompt versioning, A/B testing, monitoring, and cost optimization practices established across the platform.
Technical Stack:
- AI: Azure OpenAI (GPT-4, text-embedding-ada-002), LangChain, LangGraph
- Retrieval: Azure AI Search, Redis semantic cache
- Pipeline: Azure Data Factory, Azure Functions, Python
- Infrastructure: Docker, Kubernetes
Outcome:
3x latency improvement through semantic caching, 40% LLM cost reduction, 85% first-response resolution in conversational AI, and 100K+ documents processed daily.
