Applied Azure AI Services Portal
Built a full-stack applied-AI portal integrating seven Azure AI services behind a Node.js gateway that keeps keys server-side, including a context-preserving Azure OpenAI chat assistant, dual-model face analysis and a language-to-Personalizer feedback pipeline.
- Role
- Full-stack engineer · AI integration & frontend
- Year
- 2021–2023
- Type
- Full-stack platforms
- Stack
- Azure OpenAI · React · TypeScript · Node.js · Express · Azure AI Language (Text Analytics) · Azure AI Face · Azure AI Speech · Form Recognizer (Document Intelligence) · Azure Personalizer · Microsoft Entra ID · MongoDB
- Azure AI services integrated
- 7
The problem
Pre-sales conversations about applied AI land better with live, hands-on demos than with slides. The portal had to expose vision, document, language, speech, decisioning and generative services through one UI, without ever exposing subscription keys to the browser.
Architecture
- 01
Client
React 18 and Vite SPA (rebuilt from React 16) captures images, webcam frames, audio, documents and text, and hosts the chat widget.
- 02
Gateway
Express routes under a stable /api/azureai/<service>/<action> contract hold all service clients and process in-memory multer uploads.
- 03
AI services
Face, Form Recognizer, Text Analytics (including Health), Personalizer, Speech, Immersive Reader and Azure OpenAI chat completions.
- 04
Token brokering
The backend issues short-lived Speech tokens and Entra ID client-credentials tokens, so browser SDKs never see subscription keys.
- 05
Decision loop
Feedback records and Personalizer rank events persisted in MongoDB, with Reward calls closing the learning loop.
Engineering highlights
01
Context-aware OpenAI chat
Built the Azure OpenAI assistant on react-chatbot-kit: full role-based history (system, user, assistant) persisted client-side and replayed to chat completions each turn, with restorable chat bubbles and hand-off from a scripted guided flow.
02
Server-side credential brokering
Subscription keys stay on the server: the backend issues short-lived Speech STS tokens and Entra ID client-credentials tokens for Immersive Reader, returning only tokens and launch parameters to the browser SDKs.
03
Language-to-decision pipeline
Engineered a feedback pipeline chaining key-phrase extraction, sentiment and opinion mining into issue categories, then ranking follow-up actions with Personalizer and persisting event IDs in MongoDB for later reward calls.
04
Dual-model face analysis
Runs two face-detection models on one in-memory buffer (detection_01 for attributes, detection_03 for masks), merges the results, and identifies people against a trained person group with recognition_04.
05
Contract-stable rebuild
Rebuilt the front end from CRA, Material-UI 4 and class components to Vite, React 18, MUI 5 and React Router 6 while keeping the backend route contract unchanged.