Multi-Tenant Document Q&A API
Built a reusable multi-tenant RAG backend in Node.js: layout-aware OCR ingestion, table-safe chunking, token-budgeted embeddings and HNSW vector indexing, serving cited conversational answers to several client front ends.
- Role
- Lead backend engineer · ingestion, retrieval & API
- Year
- 2023–2024
- Type
- RAG & search
- Stack
- Azure OpenAI · Azure AI Search · Node.js · Express · LangChain JS · Document Intelligence · Redis vector store · Azure Blob Storage · Cosmos DB · Azure App Service
The problem
Teams kept policies, brochures, spreadsheets and scanned PDFs scattered across storage, and finding an answer meant opening files one by one. One backend had to serve multiple client deployments, each seeing only its own documents, with cited answers and follow-up suggestions.
Architecture
- 01
Upload
Files land in tenant-scoped Blob Storage with progress tracking and no-overwrite protection; Markdown and text converted to citable PDF.
- 02
Parse
Document Intelligence layout output rebuilt into reading-order page text with tables rendered inline as HTML.
- 03
Chunk & embed
Sentence-aware, table-safe chunks embedded in batches capped by item count and tiktoken budget, with retry and back-off.
- 04
Index
Chunks upserted into an HNSW cosine vector index with semantic configuration, tagged with page-level source references.
- 05
Answer
History-aware question rewriting, retrieval and grounded generation with filename citations, follow-up questions and fresh SAS links.
Engineering highlights
01
Layout-aware page reconstruction
Built a character map of each page's spans to locate table regions, replacing them in reading order with HTML tables preserving row and column spans, so tabular data survives into retrieval intact.
02
Table-safe chunking
Roughly 1,000-character chunks with overlap snap to sentence boundaries; if a chunk ends inside an unclosed table, the next starts at the table's beginning, so tables are never split mid-row.
03
Origin-based tenant routing
A per-environment map binds each allowed front-end origin to its own storage account and container; upload, listing, SAS generation and citation lookup all resolve through it, isolating tenants on one backend.
04
Idempotent vector writes
Chunk keys are SHA-1 hashes of source URL and chunk index, so re-uploads overwrite rather than duplicate. Removal pages through chunks by source filter until none remain.
05
Rate-limit-aware embedding
Embedding requests are grouped by item count and tiktoken budget, with configurable retries and long back-off windows, and index uploads are batched to keep large ingestions within service quotas.