Queue and steer
What happens to a message that arrives while its conversation is busy. One contract covers direct HTTP, async HTTP, WebSocket and channel ingress (issue #71). To use it, read Conversations. This page is for changing it.
Where it lives
| File | Owns |
|---|---|
packages/convex/runtimeIngress.ts | The coordinator: accept, stepBoundary, applySteering, takeNext, settle, stopOwner, maintain, recoverQueued, acquireClear |
apps/core/src/harness/ingress.ts | Candidate and delivery types, limits and TTLs, admission helpers |
apps/core/src/harness/harness.ts | prepareStep, where a steer joins a running turn |
apps/core/src/harness/ai-sdk-harness/session.ts | steerHarnessTurn, where a steer joins a running config.harness turn |
apps/core/src/harness/handler.ts | HTTP, async and channel admission, the drain loops |
apps/core/src/harness/responses.ts | The 409 and 429 admission answers, the 202 bodies, SSE error frames |
apps/core/src/harness/worker-queue.ts | The worker slots, the FIFO queue behind them, lease renewal while a run waits |
apps/core/src/shared/commands.ts | /steer, /queue, /stop and /cancel, /new and /clear, /compact |
apps/gateway/src/agent.ts | WebSocket execute, control, attach and cancel, ack and status relay |
verification/Broods/Ingress.lean | The envelope lifecycle model and its proofs. Change the mutations, update the model. |
Modes
| Mode | Busy conversation | Merged at drain |
|---|---|---|
reject | Refused, nothing stored. 409 conversation_busy. | n/a |
followup | One FIFO envelope, runs as its own turn | no |
collect | Queued like followup | the contiguous collect run at the head, one turn |
steer | Offered at the next step boundary of the running turn, else runs as a follow-up | the contiguous steer run at the head, one turn |
Every surface defaults to steer. Channel /queue <text> is a one-message followup. A merged group stops at a mode change and at a different sender (delivery.identity.userId), so in a channel a stranger's message waits for its own turn and its own policy check. A merged turn shows the model one input, but every envelope keeps its own status row and the application row lists contributingEventIds in order.
Steering is a boundary, not an abort. It never interrupts a model call or a running tool. The gateway cancel frame only detaches a reader. Hard cancellation would need its own ownership, cleanup and billing rules and is not part of this contract.
Admission
accept decides in one Convex transaction. Authentication and parsing happen before it and store nothing.
- The idempotency identity is
(accountId, agentId, scopedConversationKey, idempotencyKey), withidempotencyKeydefaulting toeventId. The binding lasts as long as the status row, seven days, so an expired envelope cannot run twice. - A recovered group comes back with the outcome, and core dispatches it (
dispatchRecoveredIngress) before handling the newcomer. activeOwnerOnlyrefuses a control that would land on a finished or different task. Subagent messages never queue on a child's conversation: a live child gets them in memory, an idle one is admitted withreject.- Each envelope stores its own execution context (resolved config ref, per-run
modeloverrides, one-turnsystemmessages), and the digest covers it, so a queued request never inherits the previous owner's overrides. deliveryholds routing ids only:http,asyncandwebsocketcarry the public event and conversation ids and, for runtime-key ingress, thepublicDeploymentIngressmarker.channelcarries the channel name, senderidentityand replysource. Never credentials, headers or message copies.
Envelope lifecycle
One runtimeIngressEnvelopes row, with the mutation that moves it. verification/Broods/Ingress.lean proves terminal states absorb and every write is fenced.
The status carries requestedMode, appliedMode and appliedToEventId. A steer that missed its boundary reads requestedMode: "steer", appliedMode: "followup". An idle request records its own event id, and an idle steer records appliedMode: "followup". A settle after /stop sets stoppedByUser. accepted and applied exist in the IngressStatus type only. The run status route overlays the async run record, so a poller can also see awaiting_approval or awaiting_input.
Ownership and fencing
The runtimeConversationCoordinators row for a conversation:
Lapsed is not a stored field: the row still names an owner, but its leaseExpiresAt has passed. Every acquisition bumps ownerGeneration, which survives lease deletion. Dequeue, history writes, status changes, result commits and release all carry it, and Convex refuses them once it moved on. Core checks it again before a tool call, a stream publish or a channel reply. A successful fenced call proves ownership for OWNER_CHECK_INTERVAL_MS (2 s), so periodic checks inside that window skip the read. Frames the client acts on (done, error, approvals, questions, structured output, waiting) always check.
Core also calls recoverQueued on boot and on a timer, for queues nobody writes to again. It pages through queued conversations one at a time, so a long queue behind a live owner hides no others.
The step boundary
Steering enters at one point, the AI SDK prepareStep hook, after every tool result of the previous step exists and before the next model call.
stepBoundary stores the finished step's messages (the first one stores the turn's own input too, so that input never costs an append of its own), checks for a stop, renews the lease once a tenth of the TTL has passed, and claims the steer prefix, all in one transaction. A turn that fails before its first boundary hands those input rows to settle or takeNext instead; one interrupted by a pod shutdown before then does not store them. A claimed steer costs one more call, the append. Subagent results queued during the pass join at the same boundary (subagents). A steer still queued when the run has no model call left is promoted by takeNext, merged with contiguous steers from the same sender, as one followup under the next generation.
A config.harness run has no prepareStep. Its steers are applied before the turn starts, and on runtimes that take mid-turn input (Claude Code, Codex, OpenCode, Pi) applySteering with textOnly claims plain-text steers at each step start and hands them over with experimental_steerTurn. A steer with an image, file or system message waits for the next turn. DeepAgents, and any runtime that refuses mid-turn input, get steers at the next turn.
HTTP
A busy request never gets a second stream. 202 always means Convex stored the envelope, never just that a worker was scheduled. Async HTTP follows the same contract.
WebSocket
The gateway delivers frames. Convex and core own admission and status, and JetStream output never counts as acceptance.
executeandcontroldefault tosteer.requestIdcorrelates frames on one socket,idempotencyKeyjoins the identity above,eventIdties the frame to the envelope.- A socket holds at most 8 controls in flight (
MAX_CONTROLS_IN_FLIGHT). One stays in flight until its status is terminal, so a steer holds its slot until the run it joined settles. One past the limit getsstatus: "failed". - A queued
executegetsackand stays open until its own run ends. The gateway reads status from core's in-cluster address, so a core that dies mid-run still ends the stream on its durable status. agentIdbecomes a NATS subject token, so the gateway refuses one with.,*,>or whitespace.- Closing the socket or sending
cancelonly detaches the reader. The core run keeps going.
Attach and replay
- A cursor is
ws-responses:<generation>:<sequence>:<event-hash>: the stream generation, the globalJsMsg.seq, and a hash of the event, so it cannot resume another event's output.afterCursoris exclusive. The client advances it only after handling a wholeoutputframe. - Without
afterCursor, replay starts at the event's earliest retained frame andreplayFromCursoris omitted. The gateway filters the conversation subject by the event id in the NATS headers. - JetStream keeps output for 3 minutes and 2,000 messages per subject. Nothing purges per event, since one subject holds several queued events. After that the Convex status, kept 7 days, is the only record. Attach never skips a gap silently.
- The SDK unwraps
outputframes foronMessageandstream(), and passes the raw envelope toonOutputfor clients that store cursors.
Channel commands
| Command | Effect |
|---|---|
/steer <text> | One steer envelope. On an idle conversation it starts a normal turn. |
/queue <text> | One followup envelope. There is no sticky per-conversation mode. |
/stop, /cancel | stopOwner: stop at the next boundary. The in-flight batch finishes. The owner settles failed with stoppedByUser, queued work moves on. |
/new, /clear | acquireClear: refused while a turn or queued envelope exists, otherwise clears history under the lease. |
/compact [instructions] | One followup envelope that runs in place of a model turn and summarizes stored history. Any HTTP, async or WebSocket run whose whole input is /compact takes the same path. |
Authorization
Authorization finishes before any envelope exists. An account key keeps its account and agent checks. A runtime key keeps its project, stage, endpoint and agent scope, and WebSocket control and attach inherit the socket's scope. Channel ingress keeps provider authentication. The server derives the scoped conversation key, so a raw conversation key, event id, status URL or NATS subject from another tenant reaches nothing. Status reads and retries repeat the checks.
Subagents and status
Steer and stop reach one conversation owner and never spread into children the parent already started. A persistent child's parent reaches it through the coordinator in memory, never through an envelope queued on its conversation. With subagent.stream: true, a child publishes on WS_RESPONSES under its own account, agent and conversation, and its taskId is the attach eventId, so attach, control and cursors work unchanged. Who may attach to a child is in subagents.
Limits
Constants in apps/core/src/harness/ingress.ts, sent to Convex on every admission. Change them together with the numbers in Conversations.
| Limit | Value | Constant |
|---|---|---|
| Queued envelopes per conversation | 100 | DEFAULT_INGRESS_MAX_COUNT |
| Queued event bytes | 1 MiB | DEFAULT_INGRESS_MAX_BYTES |
| Queued envelope lifetime | 15 min | DEFAULT_INGRESS_TTL_MS |
| Conversation lease | 15 min | DEFAULT_CONVERSATION_LEASE_TTL_MS |
| Status and idempotency records | 7 days | DEFAULT_INGRESS_STATUS_TTL_MS |
Observability
Metrics, logs and traces may record account and agent ids, event ids, a hashed conversation identity, requested and applied mode, status, queue depth, event count, age, boundary latency, fallback reason and appliedToEventId. Never message content, tool inputs or results, system prompts, credentials, delivery secrets, idempotency keys or raw headers. Tests live in packages/convex/tests/runtimeIngress.test.ts and the core ingress tests.