The big idea
Every project I touched needed an AI layer, and every integration looked the same: brittle, vendor-locked, and re-implemented from scratch. The thesis: the agent should be a reusable platform service.
Architecture: adapters in, providers out
providers/— One module per LLM vendor (OpenAI, Claude, HuggingFace) implementing a normalisedchat()interface.adapters/— One module per host application, declaring its function catalog and handlers.core/— The orchestrator that loads the right adapter, calls the chosen provider, executes tool calls, and loops until done.middleware/— JWT auth, rate limiting, audit hooks via Winston.routes/—GET /api/agent/applications,GET /api/agent/:appId/catalog,POST /api/agent/:appId/chat.
Innovative features
- App-scoped function catalogs discoverable at runtime.
- Provider-agnostic tool calling — OpenAI tools, Anthropic tool_use, HuggingFace quirks normalised.
- Per-session audit trail of prompts, providers, tool calls, results, and final responses.
- WebSocket streaming for token-by-token playback.
- File-aware uploads via Multer for multimodal flows.
- Hot-swappable provider per request.
Outcome
- One backend serves AI features across multiple internal apps without per-project rewrites.
- Switching providers is a single request-level config change.
- Full audit trail makes it operationally safe to expose to real users.