Skip to content
← All workProof of concept

Multi-Agent Advisory Workspace

Engineered a multi-agent AutoGen workspace serving five specialised agents behind one WebSocket chat, combining tool calling, hybrid-search grounding, deterministic code-side validation and per-user persisted team state for audit and deal-advisory work.

Role
Lead backend engineer · agents, prompts & state
Year
2025
Type
Multi-agent
Stack
AutoGen AgentChat · Azure OpenAI · Azure AI Search · Python · FastAPI · WebSockets · React · TypeScript · Document Intelligence · pandas · python-docx
specialised agents
5
audit evidence categories
~40
architecture iterations
3

The problem

Audit and deal-advisory teams at an advisory firm repeat rule-bound, document-heavy tasks: classifying client evidence, finding the right internal specialist, writing minutes, drafting due-diligence SOWs and normalising trial balances exported from different accounting packages. The rules lived in checklists, templates and people's experience, and every output had to land in an exact house format.

Architecture

  1. 01

    Ingest

    Attachments converted to Markdown text before reaching the model, via Document Intelligence layout extraction and later MarkItDown.

  2. 02

    Session

    A FastAPI WebSocket resolves user, session and agent, then restores the matching AutoGen team state with load_state().

  3. 03

    Orchestrate

    Each turn builds a RoundRobinGroupChat around a config-driven AssistantAgent with a 25-message buffered context at temperature 0.1.

  4. 04

    Act

    Agents call tools for hybrid vector + semantic search, Word template generation or a pandas-validated trial-balance CSV writer.

  5. 05

    Terminate & persist

    TextMessageTermination ends the turn at the agent's first text reply; save_state() checkpoints per user, session and agent.

  6. 06

    Deliver

    Markdown answers return over the WebSocket, generated documents as download links, and a lifespan task purges sessions after seven days.

Engineering highlights

  • 01

    Config-driven agent topology

    Five agents defined in one config registry (system prompt, tools, reflect_on_tool_use). Each turn spins up a single-agent team with name-based termination, so tool calls resolve and control returns to the user deterministically.

  • 02

    Validation enforced in code

    The trial-balance agent maps columns strictly by header or position into hidden structured JSON; the tool, not the model, enforces the debit/credit balance check in pandas and emits a zero-padded, four-column CSV.

  • 03

    Isolated, persisted agent state

    Every agent gets a fresh 25-message buffered context per team, so no history leaks across agents. Team state is serialised per user, session and agent after each turn and expired after seven days.

  • 04

    Grounded routing with citations

    The service-line routing agent answers only from hybrid keyword + vector retrieval with semantic reranking over firm documents, embedded with text-embedding-3-small, and cites the source file for every specialist it recommends.

  • 05

    Prompt-to-tool contracts

    The minutes agent renders reviewable Markdown, then on export emits a fixed JSON schema that a python-docx tool maps onto template placeholders and table rows, with the prompt requiring both forms to match exactly.

  • 06

    Evidence classification

    Classifies uploaded client files against about 40 audit evidence categories, returning a verdict plus ranked alternatives with likelihood percentages, and supports a filter-driven mode that tests a single subcategory.

Next projectInvoice and Expense Audit Assistant