Skip to main content

Queue and steer

What happens to a message that arrives while its conversation is busy. One contract covers direct HTTP, async HTTP, WebSocket and channel ingress (issue #71). To use it, read Conversations. This page is for changing it.

Where it lives​

FileOwns
packages/convex/runtimeIngress.tsThe coordinator: accept, stepBoundary, applySteering, takeNext, settle, stopOwner, maintain, recoverQueued, acquireClear
apps/core/src/harness/ingress.tsCandidate and delivery types, limits and TTLs, admission helpers
apps/core/src/harness/harness.tsprepareStep, where a steer joins a running turn
apps/core/src/harness/ai-sdk-harness/session.tssteerHarnessTurn, where a steer joins a running config.harness turn
apps/core/src/harness/handler.tsHTTP, async and channel admission, the drain loops
apps/core/src/harness/responses.tsThe 409 and 429 admission answers, the 202 bodies, SSE error frames
apps/core/src/harness/worker-queue.tsThe worker slots, the FIFO queue behind them, lease renewal while a run waits
apps/core/src/shared/commands.ts/steer, /queue, /stop and /cancel, /new and /clear, /compact
apps/gateway/src/agent.tsWebSocket execute, control, attach and cancel, ack and status relay
verification/Broods/Ingress.leanThe envelope lifecycle model and its proofs. Change the mutations, update the model.

Modes​

ModeBusy conversationMerged at drain
rejectRefused, nothing stored. 409 conversation_busy.n/a
followupOne FIFO envelope, runs as its own turnno
collectQueued like followupthe contiguous collect run at the head, one turn
steerOffered at the next step boundary of the running turn, else runs as a follow-upthe contiguous steer run at the head, one turn

Every surface defaults to steer. Channel /queue <text> is a one-message followup. A merged group stops at a mode change and at a different sender (delivery.identity.userId), so in a channel a stranger's message waits for its own turn and its own policy check. A merged turn shows the model one input, but every envelope keeps its own status row and the application row lists contributingEventIds in order.

Steering is a boundary, not an abort. It never interrupts a model call or a running tool. The gateway cancel frame only detaches a reader. Hard cancellation would need its own ownership, cleanup and billing rules and is not part of this contract.

Admission​

accept decides in one Convex transaction. Authentication and parsing happen before it and store nothing.

  • The idempotency identity is (accountId, agentId, scopedConversationKey, idempotencyKey), with idempotencyKey defaulting to eventId. The binding lasts as long as the status row, seven days, so an expired envelope cannot run twice.
  • A recovered group comes back with the outcome, and core dispatches it (dispatchRecoveredIngress) before handling the newcomer.
  • activeOwnerOnly refuses a control that would land on a finished or different task. Subagent messages never queue on a child's conversation: a live child gets them in memory, an idle one is admitted with reject.
  • Each envelope stores its own execution context (resolved config ref, per-run model overrides, one-turn system messages), and the digest covers it, so a queued request never inherits the previous owner's overrides.
  • delivery holds routing ids only: http, async and websocket carry the public event and conversation ids and, for runtime-key ingress, the publicDeploymentIngress marker. channel carries the channel name, sender identity and reply source. Never credentials, headers or message copies.

Envelope lifecycle​

One runtimeIngressEnvelopes row, with the mutation that moves it. verification/Broods/Ingress.lean proves terminal states absorb and every write is fenced.

The status carries requestedMode, appliedMode and appliedToEventId. A steer that missed its boundary reads requestedMode: "steer", appliedMode: "followup". An idle request records its own event id, and an idle steer records appliedMode: "followup". A settle after /stop sets stoppedByUser. accepted and applied exist in the IngressStatus type only. The run status route overlays the async run record, so a poller can also see awaiting_approval or awaiting_input.

Ownership and fencing​

The runtimeConversationCoordinators row for a conversation:

Lapsed is not a stored field: the row still names an owner, but its leaseExpiresAt has passed. Every acquisition bumps ownerGeneration, which survives lease deletion. Dequeue, history writes, status changes, result commits and release all carry it, and Convex refuses them once it moved on. Core checks it again before a tool call, a stream publish or a channel reply. A successful fenced call proves ownership for OWNER_CHECK_INTERVAL_MS (2 s), so periodic checks inside that window skip the read. Frames the client acts on (done, error, approvals, questions, structured output, waiting) always check.

Core also calls recoverQueued on boot and on a timer, for queues nobody writes to again. It pages through queued conversations one at a time, so a long queue behind a live owner hides no others.

The step boundary​

Steering enters at one point, the AI SDK prepareStep hook, after every tool result of the previous step exists and before the next model call.

stepBoundary stores the finished step's messages (the first one stores the turn's own input too, so that input never costs an append of its own), checks for a stop, renews the lease once a tenth of the TTL has passed, and claims the steer prefix, all in one transaction. A turn that fails before its first boundary hands those input rows to settle or takeNext instead; one interrupted by a pod shutdown before then does not store them. A claimed steer costs one more call, the append. Subagent results queued during the pass join at the same boundary (subagents). A steer still queued when the run has no model call left is promoted by takeNext, merged with contiguous steers from the same sender, as one followup under the next generation.

A config.harness run has no prepareStep. Its steers are applied before the turn starts, and on runtimes that take mid-turn input (Claude Code, Codex, OpenCode, Pi) applySteering with textOnly claims plain-text steers at each step start and hands them over with experimental_steerTurn. A steer with an image, file or system message waits for the next turn. DeepAgents, and any runtime that refuses mid-turn input, get steers at the next turn.

HTTP​

A busy request never gets a second stream. 202 always means Convex stored the envelope, never just that a worker was scheduled. Async HTTP follows the same contract.

WebSocket​

The gateway delivers frames. Convex and core own admission and status, and JetStream output never counts as acceptance.

  • execute and control default to steer. requestId correlates frames on one socket, idempotencyKey joins the identity above, eventId ties the frame to the envelope.
  • A socket holds at most 8 controls in flight (MAX_CONTROLS_IN_FLIGHT). One stays in flight until its status is terminal, so a steer holds its slot until the run it joined settles. One past the limit gets status: "failed".
  • A queued execute gets ack and stays open until its own run ends. The gateway reads status from core's in-cluster address, so a core that dies mid-run still ends the stream on its durable status.
  • agentId becomes a NATS subject token, so the gateway refuses one with ., *, > or whitespace.
  • Closing the socket or sending cancel only detaches the reader. The core run keeps going.

Attach and replay​

  • A cursor is ws-responses:<generation>:<sequence>:<event-hash>: the stream generation, the global JsMsg.seq, and a hash of the event, so it cannot resume another event's output. afterCursor is exclusive. The client advances it only after handling a whole output frame.
  • Without afterCursor, replay starts at the event's earliest retained frame and replayFromCursor is omitted. The gateway filters the conversation subject by the event id in the NATS headers.
  • JetStream keeps output for 3 minutes and 2,000 messages per subject. Nothing purges per event, since one subject holds several queued events. After that the Convex status, kept 7 days, is the only record. Attach never skips a gap silently.
  • The SDK unwraps output frames for onMessage and stream(), and passes the raw envelope to onOutput for clients that store cursors.

Channel commands​

CommandEffect
/steer <text>One steer envelope. On an idle conversation it starts a normal turn.
/queue <text>One followup envelope. There is no sticky per-conversation mode.
/stop, /cancelstopOwner: stop at the next boundary. The in-flight batch finishes. The owner settles failed with stoppedByUser, queued work moves on.
/new, /clearacquireClear: refused while a turn or queued envelope exists, otherwise clears history under the lease.
/compact [instructions]One followup envelope that runs in place of a model turn and summarizes stored history. Any HTTP, async or WebSocket run whose whole input is /compact takes the same path.

Authorization​

Authorization finishes before any envelope exists. An account key keeps its account and agent checks. A runtime key keeps its project, stage, endpoint and agent scope, and WebSocket control and attach inherit the socket's scope. Channel ingress keeps provider authentication. The server derives the scoped conversation key, so a raw conversation key, event id, status URL or NATS subject from another tenant reaches nothing. Status reads and retries repeat the checks.

Subagents and status​

Steer and stop reach one conversation owner and never spread into children the parent already started. A persistent child's parent reaches it through the coordinator in memory, never through an envelope queued on its conversation. With subagent.stream: true, a child publishes on WS_RESPONSES under its own account, agent and conversation, and its taskId is the attach eventId, so attach, control and cursors work unchanged. Who may attach to a child is in subagents.

Limits​

Constants in apps/core/src/harness/ingress.ts, sent to Convex on every admission. Change them together with the numbers in Conversations.

LimitValueConstant
Queued envelopes per conversation100DEFAULT_INGRESS_MAX_COUNT
Queued event bytes1 MiBDEFAULT_INGRESS_MAX_BYTES
Queued envelope lifetime15 minDEFAULT_INGRESS_TTL_MS
Conversation lease15 minDEFAULT_CONVERSATION_LEASE_TTL_MS
Status and idempotency records7 daysDEFAULT_INGRESS_STATUS_TTL_MS

Observability​

Metrics, logs and traces may record account and agent ids, event ids, a hashed conversation identity, requested and applied mode, status, queue depth, event count, age, boundary latency, fallback reason and appliedToEventId. Never message content, tool inputs or results, system prompts, credentials, delivery secrets, idempotency keys or raw headers. Tests live in packages/convex/tests/runtimeIngress.test.ts and the core ingress tests.