Proposed architecture · 16 November 2026 · Revision 3
Download MarkdownOpen full-size diagramEditable Povio diagram

Agents Hub: main-thread orchestration and delegated execution

Status: implementation specification, revision 3. Product requirements clarified; ready for implementation planning. No application changes or deployment performed. Backend review: 2026-11-15 UTC. Requirements revised: 2026-11-16 UTC.

Revision 3 confirms the working indicator appears for one or more active sub-tasks (activeCount >= 1). Revision 2 superseded the earlier working-indicator, SSE activity-feed, Hub task-metadata storage, and outbound message-queue recommendations. The previously published prototype video illustrates revision 1 and is not the acceptance reference for these changed behaviors.

Decision

Run the main conversation on Robodev with a restricted orchestrator profile. Give it only task-management tools and structured communication with the person. Run research, integrations, terminal commands, file operations, SSH, browsing and coding in worker sub-tasks. Keep the execution transcript authoritative in Robodev. Agents Hub stores a small projection of the main conversation, interaction cards, conversation/session references. Task lists, task state, task outcomes and execution details live in Robodev; Hub does not persist them.

The runtime may register all available capabilities centrally, but it must expose and authorize them separately for each role. Do not hand every tool or credential to the main thread and rely on its instructions to delegate. Workers receive their own scoped capabilities from a broker; the main thread never receives the workers' bearer tokens, SSH credentials or vault contents.

Editable Povio diagram

Proposed architecture

What the deployed code does today

Inspection scope

Agents Hub was inspected inside production container mission-control-api-1, image ghcr.io/povio/mission-control:5b3b34e9547ec761deea9e791abe4777eabc2e39. Its backend TypeScript was recovered from the deployed JavaScript source maps. Robodev was inspected inside staging container robodev-agents-staging-api-1, image robodev-agents-staging/api:b8f253e. The generic /opt/robodev-agents-src directory is older, so findings below use the deployed container snapshot instead.

This review traced backend persistence, task handling, event contracts and harness configuration. The frontend source was not included in the deployed backend source snapshot; exact frontend component names and existing hover behavior still need repository inspection during implementation. No private conversation rows or credential values were needed for this review.

Current ownership and duplication

Data Current code path Current storage / delivery
Execution prompts and results Robodev Prisma Session / Turn Robodev PostgreSQL
Normalized execution events Robodev recorder.ts, Prisma Event Robodev PostgreSQL, paginated history and SSE
Raw harness output Robodev recorder, Prisma RawLine Robodev PostgreSQL
In-progress partial text Robodev recorder partial() Live only; partial events themselves are not persisted
Main case messages Hub WorkflowCaseAgentMessage Hub PostgreSQL; parsed reply, status, questions and connection requests
Tool-call summaries Hub turn service toolCalls() / summariseToolCalls() Copied from Robodev into Hub toolCalls JSON; up to 50 calls with displayed input summaries
Run capture Hub persistRun() → persistAiAgentRun() Hub WorkflowAiAgentRun.stdout/stderr/error; Robodev polling capture contains the last response body, including result/usage data, rather than necessarily the full event stream
Detailed session stream Hub WorkflowCaseAgentStreamService / proxyRobodevStream() Authorized proxy to Robodev; payloads trimmed before forwarding
Coding task state / result Hub WorkflowCodingTaskService Hub task metadata and encrypted child result; report() also inserts up to 30,000 characters of child output as a case event and queues a main reply
Personal-agent tool summaries Hub AgentRuntimeService.rememberTools() Also copied to Hub Slack-message history; separate from case chat

Answer to the storage question: it is mixed today. Robodev owns the full execution history, but Hub independently persists chat messages, tool summaries, run captures and coding-task results. Details can also be streamed from Robodev. The implementation is not currently “everything lives only on staging and is fetched only when Logs opens.” The code confirms these write paths; this review did not audit historical database contents or prove how often every frontend opens a stream.

Existing pieces to reuse

Desired product behavior

The conversation contains user messages, main-thread assistant messages, questions, connection prompts and secret-request forms. Worker output, reasoning, raw errors, commands and tool calls never become ordinary chat bubbles automatically. A worker can request an interaction; the orchestrator or a trusted UI projection presents the structured form in the main conversation.

The composer and Send action remain available regardless of main-thread execution, sub-task activity, an outstanding request, a prior undelivered message, or a previous error. Each user submission appears immediately with its own delivery indicator. No “wait for a turn” or “agent busy” gate is allowed.

Agent is working is exclusively a sub-task activity indicator. It is never driven by main-thread generation, message delivery, pending parent summaries or the availability of the composer. Use real-time WebSocket events from the Robodev session. At zero active sub-tasks it is hidden even while the main thread generates. At one or more active sub-tasks it is visible and clickable: showWorking = activityIsFresh && activeCount >= 1. This threshold was explicitly confirmed by the owner. Main-thread activity never contributes to the count.

For the initial implementation, count workers in provisioning, running or cancelling as active; queued and waiting-for-user/secret tasks appear in the task list without being labelled working. Keep waiting interaction cards visible independently. A disconnected feed displays a separate “Reconnecting activity” state; an unknown count must not be presented as zero or as a fresh authoritative working state.

Clicking the activity indicator opens a task drawer listing all active tasks and paginated completed/failed/cancelled history from Robodev. Include an always-accessible “Sub-tasks” action so history remains reachable when the working indicator is hidden. Every task can be opened for details; fetch both the list and individual details from Robodev through a stateless authorized Hub proxy. Store neither in the mission-control database. Reveal a small Logs action on assistant-row or activity-row hover. Also reveal it on keyboard focus and provide a tap/menu equivalent on touch devices. Hover only reveals the control; clicking opens a drawer and starts the details request. No trace prefetch on hover or page open. Closing the drawer aborts its event stream. Keep questions and secret forms available in normal chat, independent of the drawer.

The drawer presents the task tree, tool calls, commands, outputs and diagnostic events, grouped by worker and turn. It can include orchestrator task-management events, which are still logs even though they are hidden from the normal conversation. Logs must not expose secret values or promise private model reasoning that the provider does not supply.

Example: “Which VPSes can you reach?”

  1. The user message is accepted with a stable id and submitted to the Robodev orchestrator.
  2. The main thread says “I’m checking access to the configured VPSes.” It starts a verification sub-task using a stable idempotency key. The UI shows Agent is working as soon as this one worker becomes active; clicking it opens the sub-task drawer.
  3. The broker starts a worker with SSH and terminal capabilities. The worker reads the configured hosts and performs the authorized checks. Commands and outputs stay in the worker history on Robodev.
  4. The worker returns a bounded structured summary and evidence references. Completion durably queues one parent continuation.
  5. The main thread reports the verified hosts and any failures in a normal message. The working indicator follows worker count only; it can disappear before the main-thread answer finishes.
  6. If the person clicks Logs, Hub authorizes the request and fetches the worker's details from Robodev. Nothing in step 6 is required for step 5 to work.

Tool and credential boundaries

These are proposed interfaces, not existing API names.

Capability Main orchestrator Worker
Start/list/inspect task status Yes, own case only Own descendants only if explicitly enabled
Send task instruction / cancel task Yes, own tasks No sibling or parent authority
Read task result Bounded structured summary / artifact references Own full result
Public message / question / secret-request reference Structured output or narrow communication tool Request routed to main UI; no raw worker bubble
Terminal, files, SSH, browser, arbitrary HTTP No As authorized for the task
Business MCP tools No Intersection of owner grants, run policy and task scope
Vault secrets and integration tokens No Broker injects only required credentials
Raw log API User-facing authenticated proxy; not a main-agent tool Own execution context as needed

Suggested management surface: start_task, list_tasks, get_task_status, get_task_result, send_task_message, cancel_task. Responses must exclude raw event pages. Public messages/questions can continue using the existing structured reply contract; no extra tool is necessary just to speak.

The broker binds owner/team, case, parent session, child task/session, role, permission ceiling and expiry to each capability token. A child result never grants new authorization. Disabled tools and workflow approval policies remain effective. Moving execution into a worker must not turn a review-required write into an automatic write. Payments remain disabled.

Reject both disallowed discovery and invocation. Reject role changes and grant widening on session resume. Distinguish an omitted allowlist from an explicitly empty allowlist: current harness checks such as allowedTools?.length must not interpret “no tools” as “all tools.” Do not mount the user's shared workspace, SSH agent or credential files into the orchestrator environment.

Runtime implementation choice

Recommended first implementation: retain the existing coding-capable runtime for workers and run a separate minimal orchestrator executor on Robodev. It can use a model tool loop that exposes only the task broker and structured reply schema. It needs no terminal, user filesystem or browser. This makes the main/worker boundary enforceable without depending on an instruction or an unverified CLI tool-filter feature.

A native Codex parent/child implementation is an optimization only after proving the deployed harness can enforce different native-tool sets for parent and child. The current nested-event mapper does not prove that. If a supported harness can meet the isolation contract, use it behind the same task API. Otherwise keep the minimal orchestrator plus separate worker sessions. Do not silently fall back to a full-access main session.

The server may know all MCP definitions and resolve them lazily for each child. Avoid putting the entire catalogue in the main prompt. Resolve relevant capabilities from a task type/description and mint child-scoped relay tokens. No need to proxy terminal/file operations through the main model: execute them in the worker sandbox and record them in its log.

Data ownership after the change

Concern Authoritative owner What Hub retains
Accepted main-thread conversation and ordering Robodev durable conversation/event feed Main-thread projection for display, unread state and search
Tool events, command output, raw harness logs Robodev No persisted payloads or task-event cursors; transient proxy only
Task lists, task tree, history, outcomes and artifacts Robodev task service / artifact storage No task rows, instructions, results, list snapshots or persisted task metadata; main-thread messages may include user-facing artifact links
Task execution state and active count Robodev session/task service In-memory WebSocket/UI state only; no durable mirror
Questions and secret requests Durable interaction records with Robodev execution linkage Public question/form metadata and request id; never submitted secret values
Workflow definitions, projects, application approval records Hub Existing business/application records
Usage totals and latency metrics Robodev execution records Optional aggregate numbers; no stdout as a usage database

Keeping main messages in Hub is intentional, as requested. They are a rebuildable projection, not a second competing execution transcript. Workers may need summaries fetched automatically for the parent to answer; “details only on request” applies to full execution logs, not task lifecycle or bounded results required to complete the work.

Immediate message submission and delivery semantics

Each Send action dispatches its own API request immediately, at the time the person submits it. Invoke requests in user-created order, without waiting for previous HTTP responses, delivery acknowledgements or model turns. Do not debounce, batch messages, or implement a client/Hub outbound queue that holds message B until A succeeds. Three quick sends produce three independent in-flight API calls.

Each submission carries clientMessageId (stable UUID), clientStreamId (browser conversation stream), monotonically increasing clientSequence, createdAt and its content. The browser invokes requests in sequence, but independent HTTP requests can arrive out of order. Robodev records server acceptance order and client sequence, and its scheduler preserves per-stream submission order without delaying acceptance or acknowledgements. A missing lower sequence must not indefinitely block later accepted messages: track the gap as undelivered, process available input, and reconcile a late message by identity without replaying executed work. Separate tabs/devices have distinct streams; use server-assigned conversation event sequence for their merged history.

Keep these per-message states visibly distinct:

Indicator Meaning
Sending Request dispatched; no acknowledgement yet
Received by Hub Optional intermediate transport acknowledgement; not delivered to the agent
Delivered Robodev has durably accepted this message, even if model processing is pending
Queued for processing Optional secondary status from Robodev; already delivered, not a delivery failure
Not delivered / retry Submission failed or delivery is uncertain; subsequent messages still send

Hub may persist main-thread message content, its stable identity and delivery acknowledgement as part of the allowed conversation projection. It forwards every attempt immediately and does not promise downstream delivery from a Hub-only acknowledgement. If Robodev is unavailable, show the affected message as undelivered and offer retry with the same id. Never mark delivered on an optimistic render. A timeout after a committed write is reconciled by id before retry; Robodev idempotency makes retries safe. Keep draft/undelivered messages recoverable as conversation data without making them a serialized outbound queue. Authentication expiry, network failure or service limits cannot be eliminated: expose the actual state, preserve the text, and never disable later sends because an earlier message failed.

Robodev accepts valid messages during active turns. Queueing for model processing is permitted inside Robodev; user submission never waits for it. Model execution can remain serial while API acceptance is concurrent. Default new messages to durable pending input at the next safe orchestration boundary; interrupt/cancel remains explicit. Record which input ids each main response addresses, so accepting several messages does not require one answer per message or lose any input.

Record public assistant output and structured interactions durably on Robodev before acknowledging projection delivery. Hub projects with (conversationId, eventId) uniqueness and a conversation cursor. This conversation cursor is distinct from a task-log cursor and is allowed in Hub.

Proposed Robodev records: conversation, public message, task, task outcome, interaction request and delivery/outbox event. Tasks link parentTaskId, orchestrator session/turn, child session/turn and capability profile. Hub retains the main conversation-to-Robodev session association only, not task-link rows. Robodev validates task membership against that session on every task list/detail request. For existing normalized execution events, preserve sessionId, turnId, itemId, parentItemId and seq. Version any added envelope fields according to Robodev AGENTS.md. Keep raw logs separate from the public conversation projection.

Lifecycle and delivery

Proposed task states: queued, provisioning, running, waiting_for_user, waiting_for_secret, cancelling, completed, failed, cancelled. Parent continuations have their own queued/running/settled state, which never contributes to the working indicator or blocks new user submissions.

Use an authenticated WebSocket subscription to the Robodev session, optionally bridged transparently through Hub to keep service credentials server-side. The authoritative snapshot and events originate in Robodev. Send a coherent initial activity.snapshot containing activeCount, active worker identities/statuses, revision, and lastEventId, then ordered task.changed, task.finished and count updates. These proposed names are contract requirements to implement, not existing endpoints.

Subscribe and snapshot without a race: obtain a durable replay cursor with the snapshot and replay subsequent events. Use event ids and task revisions to deduplicate, reject stale updates and recover after disconnect. On reconnect request replay after the last in-memory event id; if expired, take a fresh snapshot. Heartbeats expose connection health. Do not derive authoritative count by blindly incrementing/decrementing on repeated events. Task events and cursors are retained by Robodev and held only in browser/Hub process memory during delivery.

Persist a Robodev outbox/event log so task completion and parent wake-up survive restart. Hub's public-conversation consumer may persist messages and interaction records, but its task WebSocket bridge must not write task payloads to database, queues, logs, analytics or durable caches. No recurring task polling is needed to drive the UI. An explicit reconnect/snapshot repairs state. Paginated task history is loaded only on drawer open or user pagination; individual execution details only on task selection. On completion, store the result first, then enqueue the parent wake-up atomically. Deduplicate with task id plus result/turn version. Serialize parent execution where required; coalesce completion wake-ups while it is busy without blocking message acceptance. Keep this scheduling state in Robodev. Resumed tasks create another result version. Store internal completion references outside the public message table; do not inject the entire child result into a visible event. The main thread receives a bounded result through Robodev context or get_task_result; Hub does not retain an intermediate full child result.

Handle active-task steering explicitly. The existing coding-task continuation refuses active turns: the new API must distinguish queued instructions, steer, interrupt and settled-task continuation, with clear acknowledgement and cancellation races. An uncertain external write is reconciled instead of blindly retried.

Bound task concurrency, nesting depth, runtime and result size. Reuse separate task workspaces; coding uses isolated Git worktrees and serialized Git metadata operations. Cancellation revokes task capabilities and stops its process/session without deleting canonical history. Do not let a late completion resurrect a cancelled task.

API and frontend changes

Proposed routes are illustrative contracts to implement, not claims about existing endpoints:

The proxy holds service credentials server-side. Apply user/case/team access checks, task membership, rate limits, size bounds and redaction. Use private/no-store cache policy for detail responses and avoid response bodies in application/access logs. Permission revocation must also terminate or revalidate long-lived streams. Browser memory may cache task lists/details while the page is open; do not persist task data or execution traces in localStorage or IndexedDB. Main-thread message recovery is a separate allowed concern.

Normal conversation serializers must stop returning tool summaries, run stdout/stderr and child completion dumps. Normal chat must not subscribe to the raw event proxy. The Logs drawer may reuse the existing normalized event renderer. Status and secret interactions must continue working while the drawer is closed.

Implementation work by area

Area Starting points in inspected code Required change
Hub conversation execution src/modules/workflows/agent-chat/workflow-case-agent-turn.service.ts Select orchestrator profile; split immediate message acceptance from turn execution; receive public projection; remove busy gate and copied summaries
Hub case persistence workflow-case-agent.store.ts, agent-chat/persistors/prisma/workflow-case-agent.prisma-repository.ts, WorkflowCaseAgentMessage Store public messages/interactions, client ids, ordering and receipts; retain session association; remove task rows/results from target chat path
Hub run history src/modules/workflows/workflow.preparation.service.ts, WorkflowAiAgentRun Keep execution references and usage aggregates; stop storing new trace/response captures for migrated sessions
Hub worker foundation src/modules/workflows/runtime/workflow-coding-task.service.ts, .mcp.ts, .request.ts Move general task registry, state, outcomes and scheduler into Robodev; optional project; replace Hub coding-task persistence for migrated conversations
Hub self-MCP / integrations src/modules/workflows/agent/workflow-agent-self-mcp.ts, workflow-agent-mcp-setup.ts, custom MCP relay Orchestrator-only tool scope; child-specific tokens, run context and existing approval semantics
Hub logs and status workflow-case-agent-stream.service.ts, workflow-agent-session-stream.service.ts, workflow-robodev-stream.ts Robodev-backed activity WebSocket and stateless task-list/detail proxy; no persisted task mirrors
Hub personal-agent parity src/modules/agents/agent-runtime.service.ts Remove rememberTools() copying when applying this design outside case chat
Hub frontend Locate in source repository during implementation Always-enabled composer, concurrent submissions, per-message receipts, sub-task-only count indicator, click-through active/history drawer
Robodev tasks and policy apps/api/src/routes.ts, runtime.ts, harness.ts, codex-app-server.ts, mcp-bridge.mjs Immediate idempotent inbox acceptance, processing scheduler, canonical task registry, enforced roles, broker, parent wake-up and replayable WebSocket
Robodev storage/contracts apps/api/prisma/schema.prisma, apps/api/src/recorder.ts, packages/events/src/envelope.ts Canonical public conversation and task records; retained execution history and versioned event changes

Hub paths in this table are original TypeScript paths recovered from deployed source maps. Existing generated .resolved.prisma files are evidence, not the files to edit. Locate their original schema fragments and create migrations in the actual repository.

Migration and rollout

  1. Add versioned task/lifecycle/public-message contracts and metadata references. Establish canonical retention and recovery before removing any Hub log copies.
  2. Add WebSocket activity delivery, concurrent message acceptance and the Robodev-backed task drawer behind a feature flag. Verify ordinary chat performs zero detailed-event requests.
  3. Deploy the capability broker and minimal orchestrator executor. Prove direct execution is denied before routing real conversations through it.
  4. Generalize coding tasks, enable no-project tasks and preserve workflow/run authorization in worker tokens. Switch one-off task cases first; migrate scheduled/workflow cases separately without changing their approval policy.
  5. Stop new tool-summary copying, full child-result chat events and migrated run captures. Switch existing history readers to execution references.
  6. Backfill conversation/session associations and canonical Robodev task membership; do not introduce new Hub task-link storage. Existing WorkflowAiAgentRun rows do not have an explicit session field in the inspected schema; recover references from captures only where verifiable. Quarantine unmappable legacy rows rather than inventing ids or deleting the only copy.
  7. Verify Robodev history is readable after sandbox stop/termination and after restoration from backups. Robodev already distinguishes ending execution from deleting history; validate retention and explicit deletion behavior for all referenced sessions/tasks.
  8. Remove duplicate legacy payloads only after reference coverage and retention are verified. Keep this as a separate, deliberate data migration. Backups follow their own retention schedule.

Rollback restores the prior routing behind the flag while retaining canonical Robodev logs and identifiers. Do not restore broad tools to a session labelled restricted as a silent fallback. A runtime incompatibility must fail closed or use the minimal orchestrator.

The staging service would become a production dependency for Hub's chat execution and log history. Give it production-grade backup, monitoring, retention and recovery guarantees before relying on it as the sole execution store. “Single source” means one authoritative logical store; replicas and backups are still required.

Acceptance criteria and validation

This deliverable was validated against deployed backend code and a rendered Povio diagram. It changes documentation and creates a diagram only; application tests and deployment are outside this proposal's completed work.

Implementation sequence and handoff

  1. Robodev backend: implement canonical message inbox and stable receipt lookup independently of harness turn execution. Add task registry/list/detail endpoints and session-scoped WebSocket snapshot/replay contracts. Preserve existing permission ceilings.
  2. Hub backend: forward message attempts immediately, persist only allowed conversation projection/receipts, and add stateless authorized task and WebSocket proxies. Remove active-turn submission rejection and any migrated task persistence or raw-result copies.
  3. Hub frontend: keep Send independent of every prior message/agent state; add per-message delivery icons and retry; connect activity WebSocket; add clickable activity indicator and always-available Sub-tasks history action; lazy-load task lists and details.
  4. Integration/migration: establish Robodev membership and retention for old task records before retiring Hub task tables for migrated conversations. Legacy data remains isolated until verified migration; no new task copies on the new path.
  5. Verify the acceptance matrix above, then update the prototype video to revision 3 before treating it as a visual acceptance reference. Deploy only within separately authorized implementation/deployment scope.

The implementation is ready to estimate and split by these workstreams. The working-indicator predicate is confirmed as activeCount >= 1; there are no outstanding product clarification questions. Other defaults are explicit here: independent immediate POSTs, Robodev-only processing queues, server acceptance receipts, WebSocket task activity, and Robodev-only task persistence.

Confirmed working-indicator behavior

The owner confirmed one or more active sub-tasks on 2026-11-16. Show the clickable Agent is working indicator whenever authoritative, fresh Robodev WebSocket state reports activeCount >= 1. Hide it at zero. Main-thread generation, message submission and pending parent summaries do not affect it. The separate Sub-tasks action always remains available for completed history, waiting tasks and zero-active states.