Aircraft Records Q&A and Validation
Engineered retrieval and extraction over scanned aircraft maintenance records: a queue-driven OCR and LLM ingestion pipeline, an identifier index with set-intersection queries, and JSON-mode spreadsheet validation returning cited dates, hours and cycles.
- Role
- Engineer · ingestion, identifier search & LLM pipeline
- Year
- 2024–2025
- Type
- RAG & search
- Stack
- Azure OpenAI GPT-4o · Azure AI Search · Document Intelligence · Python · Quart · Node.js · Azure Functions · Cosmos DB · React · TypeScript · Entra ID · MSAL
- spreadsheet event layouts validated
- 5
- identifier types extracted per page
- 8
The problem
Before an aviation group can induct an aircraft, records staff must evidence hundreds of maintenance events with citations from thousands of scanned pages. The same item appears under many aliases, and finding last-performed dates, hours and cycles by hand slowed fleet phase-in.
Architecture
- 01
Ingest
Queue-triggered Function parses each PDF page with Document Intelligence, rate-limited and processed sequentially with backoff on embedding calls.
- 02
Extract
GPT-4o pulls AD, SB, task card, work order, serial and part identifiers plus a calendar-validated most-recent date per page.
- 03
Index
Chunks go to a per-aircraft hybrid vector index and to per-identifier Cosmos DB tables keyed by identifier and chunk.
- 04
Retrieve
Chat uses tool-calling query rewriting over hybrid semantic search; identifier queries intersect partition lookups and rank newest first.
- 05
Validate
Each spreadsheet event's top candidate pages go to GPT-4o in JSON mode, returning date, hours, cycles and a source citation.
Engineering highlights
01
Identifier index with intersections
Built a Cosmos DB Table identifier index: identifier as partition key, chunk as row key. Multi-identifier queries become set intersections, prefix searches become computed range scans, and dense parent documents are ranked down.
02
Cited spreadsheet validation
Five spreadsheet layouts parsed into boosted queries and identifier partitions. Candidates are filtered by event name and date, then validated per page in JSON mode with explicit synonyms for hours, cycles and dates.
03
Throttle-safe ingestion
Built the first ingestion Function: small queue batches, sequential paging, exponential backoff honouring retry-after, blob metadata markers preventing re-indexing, and a regex calendar check guaranteeing a valid date for date-ordered retrieval.
04
GPT-4o migration
Migrated the chat backend to GPT-4o, switching search-query rewriting to tool calling and adding token-limit-aware history trimming, with iterative domain-prompt tuning around aviation alias vocabulary.
05
Per-aircraft tenant isolation
One admin call provisions a project's vector search index, blob container and Cosmos tables; every request then resolves its index and tables from the selected aircraft, isolating each fleet asset's records.