Low-Latency Live Agent WebSocket Streaming Engine
Agent Mission Control
Autonomous AI agents usually run as black boxes. An operator deploys them, walks away, and finds out afterwards that the agent spent a fortune, made an unsanctioned outbound call, or silently failed. This is a control plane for operating them instead: deploy a mission, watch the reasoning arrive in real time, intercept risky actions before they happen, and review the actual cost and latency afterwards.
Technology
The problem
Running an LLM tool-calling loop for several minutes creates three problems that ordinary request/response software does not have. First, the intermediate states are invisible — the operator cannot see what the agent is reasoning about while it happens. Second, the agent can take actions with real consequences without anyone authorising them. Third, the token spend is unaccounted for, so cost is discovered on the invoice rather than managed during the run.
Approach
- 01
Chose Django Channels and Daphne so HTTP and WebSockets are served by a single ASGI process, with the ORM kept behind sync-safe boundaries. One process means one place for auth, routing and error handling instead of a second realtime service to keep consistent.
- 02
Made the loop itself observable. Every thought, tool call and status transition is persisted and broadcast over a per-mission channel group, so a reconnecting client catches up from stored state rather than starting blind.
- 03
Implemented a human-in-the-loop gate. Tools marked as privileged pause the loop, emit an approval prompt, and wait for a decision. Timeouts fail closed — an unanswered approval is a rejection, not an implicit yes. This is the part that makes autonomy safe to operate.
- 04
Instrumented the run honestly. Token estimates and per-tool durations are captured at each step and rolled up into mission totals, then into per-agent and per-tool aggregates, so the cost dashboard is computed from real step data rather than an estimate applied after the fact.
- 05
Moved persistence to PostgreSQL under concurrent write load. Socket traffic, REST polling and worker writes all serialise badly on a single-file SQLite store, so the schema, migrations and a least-privilege application role replaced it.
- 06
Made the interface fail gracefully. Every page has loading, empty and error states, polling is stale-while-revalidate, and an error boundary turns a dropped backend into a banner instead of a blank screen.
Architecture
- 01
React SPA
A Vite + React mission-control interface. It takes one REST payload per page and upgrades to a live socket only where liveness actually matters — the agent feed.
- 02
HTTP surface (DRF)
Deploy a mission, list missions and steps, read agent and tool aggregates, toggle tool availability, and read dashboard metrics.
- 03
WebSocket consumer
A Channels consumer joins a per-mission group and streams step upserts, status transitions and approval prompts as they occur, with persisted state for reconnect.
- 04
Agentic loop
A tool-calling loop with iteration budgets and a hard step cap. Standard tools execute immediately; privileged tools suspend the loop and await a human decision.
- 05
Approval gate
Approve resumes and executes; reject or timeout skips the tool and the loop continues. Every decision is persisted as a step outcome.
- 06
Telemetry & store
Per-step tokens and duration roll up into mission cost, per-tool call counts and average latency, then into per-agent totals — all in PostgreSQL.
Standard tools run. Privileged tools block on a human. An unanswered approval is a rejection, not an implicit yes.
Key features
Challenges
Streaming the internals of a long-running loop
A tool-calling run lasts minutes and its intermediate states are the whole point of the interface. Each state had to be persisted and broadcast, not just printed, so that a client that reconnects mid-run can reconstruct what happened rather than joining a blank stream.
Pausing an async loop without deadlocking it
A privileged tool has to suspend the agent, wait for a decision that might arrive seconds later or never arrive, and then resume with intact state. Doing that naively deadlocks the worker; the decision is instead persisted as a step the loop is waiting on, with a timeout that resolves it.
Making cost data trustworthy
Post-hoc estimates are guesses with extra steps. Token counts and durations are captured at each step as it happens and aggregated from stored step rows, so the dashboard is a query over recorded facts instead of a calculation applied to an assumed average.
Concurrent writes across three access paths
Socket consumers, REST views and background workers all write. A single-file database serialised these into lock contention, which is why the store moved to PostgreSQL with proper migrations and a role scoped to the one database it needs.
What I learned
A human approval gate is not a pause button — it is a state machine. The safe version treats silence as a negative decision, because the failure mode of an unattended approval is an action nobody authorised, and that is not a failure you can undo.
Scope & current state
Being precise about scope
- Demonstrated end to end against a live tool-calling model, including a full run that hit a privileged tool, paused, was approved, and completed.
- Telemetry figures reported in the project README come from seeded demo data and local verification runs, not from production traffic.