Runner terminal outcomes and exits

Source-build preview for v0.40.0. The terminal-result capability is now implemented for exact captured inline text units. Copilot/text execution additionally requires inline.copilot-text, an explicit model and paid consent; terminal capability alone does not enable it. Archives and container publication are not enabled by this capability.

Run and save the actual response

waza runner capabilities
waza runner validate < examples/runner/unit.json
waza runner run < examples/runner/unit.json > terminal.json

Only run starts a runtime. It emits one compact JSON line with kind waza.runner-terminal and independent terminal schema_version: "1.0". Inspect the outcome and exit; do not treat an input-derived identity or an HTTP 200 as success. The selected trial/attempt is executed once, without discovery, cache, automatic retry or adjacent work.

For HTTP, submit the same complete JSON to POST /api/v1/runner/run:

curl --silent --show-error \
  -H 'Content-Type: application/json' \
  --data-binary @examples/runner/unit.json \
  http://127.0.0.1:3000/api/v1/runner/run

A complete request returns HTTP 200 even for evaluation, admission or runtime failure; interpret outcome and exit_code. The X-Waza-Exit-Code response header repeats that code. Input transport errors use HTTP 400, invalid media 415, oversized body 413, interrupted/timed-out body 408, and unsupported deadline transport 500, with the same terminal body when writable.

Outcomes and operator action

Outcome	Exit	Meaning and action
`success`	0	A complete passed observation contains every admitted check. Inspect execution mode; this is not independently witnessed model assurance.
`evaluation_failure`	1	At least one observed check failed. Inspect the explicit verdicts; do not relabel it an infrastructure retry.
`infrastructure_failure`	2	Startup, execution, shutdown, terminalization or delivery failed. Inspect the boundary and diagnostic; a completed observation cannot override this failure.
`cancelled`	3	Cancellation was observed, including SIGINT/SIGTERM during blocked stdin or active HTTP request shutdown.
`timeout`	4	An input, request or execution deadline expired. Inspect the boundary before changing a budget.
`unsupported_capability`	5	Required capability or runner version is unavailable. `unsupported_version` distinguishes a version rejection. Negotiate supported requirements; consent alone never enables a profile.
`invalid_input`	6	JSON/schema, captured source, exact selection, flags or arguments are invalid. Correct the request rather than dropping unsupported semantics.

failure_boundary is none only for completed evaluation verdicts; other boundaries identify input, negotiation, validation, identity, startup, execution, shutdown, output or terminalization. Input rejection has zero engine calls and no identity/selection/observation. After admission, retain the actual canonical identity even on failure. It names input, not successful execution or delivery.

Typed deadline causes take precedence over cancellation, which takes precedence over an ordinary runtime error. An eval/task execution deadline produces timeout / exit 4 at the execution boundary even while the outer request budget is still active. A task’s timeout_seconds overrides the eval timeout; the request’s timeout_ms still bounds the complete invocation. The error observation retains the admitted identity, selection, trial/attempt and actual engine call count, without inventing grader verdicts. Error text alone does not establish a typed timeout or cancellation. A later cleanup error is not swallowed: its boundary is reported and an earlier passed/failed observation remains recorded. Incomplete, malformed or oversized evidence cannot become success.

Schema and evidence limits

Download the terminal envelope 1.0 schema. It is independent of request and identity algorithm versions. Unknown fields, duplicates, malformed Unicode, wrong scalar types, contradictory outcome/exit pairs and invalid presence fail closed. Schema shape alone is not proof that the identity was derived or that execution occurred.

observation is a bounded projection of the actual returned attempt, not a full RunResult or authored expected response. It includes mock or copilot mode, call count, bound selection, run_number, selected attempts, status, explicit grader verdicts and an error for an error observation. The production attempts field is the selected ordinal (example: 2), not a newly invented local retry count. The complete admitted grader set includes eval/task graders and applicable text expectations. No transcript, final text, score or billing totals are asserted. Unavailable usage is not zero; explicit verdict false remains false.

Limits are 1 MiB per envelope, 16 diagnostics, 4096 UTF-8 bytes per message, 128 summarized graders and 256-byte grader/selection names. Overlong rejection messages explicitly mark truncation; excess rejection diagnostics have an explicit remaining count. Unrepresentable observed evidence yields infrastructure_failure / terminalization with unrepresentable_evidence; the identity, observed call count and original classification survive, while the missing summary is explicitly diagnosed.

The following are synthetic schema examples, not execution receipts or live assurance. Each passes the same schema and contextual validator; actual producer conformance runs through CLI/HTTP and the frozen admitted identity vectors.

success (exit 0)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "success",
  "exit_code": 0,
  "failure_boundary": "none",
  "validation": "valid",
  "engine_calls": 1,
  "selection": {
    "shard": "only",
    "task": "task",
    "trial": 2,
    "attempt": 3
  },
  "identity": {
    "version": "sha256-waza-c14n-v1",
    "canonical_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "run": "run-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "shard": "shard-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "task": "task-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "trial": "trial-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "attempt": "attempt-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  },
  "observation": {
    "execution_mode": "mock",
    "engine_calls": 1,
    "selection": {
      "shard": "only",
      "task": "task",
      "trial": 2,
      "attempt": 3
    },
    "run_number": 2,
    "attempts": 3,
    "status": "passed",
    "validations": [
      {
        "identifier": "text",
        "passed": true
      }
    ]
  },
  "diagnostics": []
}

evaluation_failure (exit 1)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "evaluation_failure",
  "exit_code": 1,
  "failure_boundary": "none",
  "validation": "valid",
  "engine_calls": 1,
  "selection": {
    "shard": "only",
    "task": "task",
    "trial": 2,
    "attempt": 3
  },
  "identity": {
    "version": "sha256-waza-c14n-v1",
    "canonical_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "run": "run-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "shard": "shard-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "task": "task-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "trial": "trial-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
    "attempt": "attempt-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  },
  "observation": {
    "execution_mode": "mock",
    "engine_calls": 1,
    "selection": {
      "shard": "only",
      "task": "task",
      "trial": 2,
      "attempt": 3
    },
    "run_number": 2,
    "attempts": 3,
    "status": "failed",
    "validations": [
      {
        "identifier": "text",
        "passed": false
      }
    ]
  },
  "diagnostics": [
    {
      "code": "evaluation_failure",
      "message": "Synthetic example: text assertion failed."
    }
  ]
}

infrastructure_failure (exit 2)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "infrastructure_failure",
  "exit_code": 2,
  "failure_boundary": "identity",
  "validation": "valid",
  "engine_calls": 0,
  "diagnostics": [
    {
      "code": "identity_error",
      "message": "Synthetic example: identity derivation failed before runtime construction."
    }
  ]
}

cancelled (exit 3)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "cancelled",
  "exit_code": 3,
  "failure_boundary": "input",
  "validation": "not_checked",
  "engine_calls": 0,
  "diagnostics": [
    {
      "code": "cancelled",
      "message": "Synthetic example: input admission was cancelled."
    }
  ]
}

timeout (exit 4)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "timeout",
  "exit_code": 4,
  "failure_boundary": "input",
  "validation": "not_checked",
  "engine_calls": 0,
  "diagnostics": [
    {
      "code": "timeout",
      "message": "Synthetic example: input admission deadline expired."
    }
  ]
}

unsupported_capability (exit 5)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "unsupported_capability",
  "exit_code": 5,
  "failure_boundary": "negotiation",
  "validation": "not_checked",
  "engine_calls": 0,
  "diagnostics": [
    {
      "code": "unsupported_version",
      "message": "Synthetic example: no mutually supported interface version."
    }
  ]
}

invalid_input (exit 6)

{
  "kind": "waza.runner-terminal",
  "schema_version": "1.0",
  "outcome": "invalid_input",
  "exit_code": 6,
  "failure_boundary": "validation",
  "validation": "invalid",
  "engine_calls": 0,
  "diagnostics": [
    {
      "code": "invalid_input",
      "message": "Synthetic example: requested task does not match the captured task."
    }
  ]
}

Cancellation, output and delivery

Input admission is bounded to 4 MiB. HTTP body reads install a five-second deadline before media-type validation and fail closed if read deadlines are unavailable; server cancellation interrupts partial uploads and waits for handler cleanup. Write-deadline support is also required before admission, checked without arming a timer. The five-second write budget starts only when terminal delivery begins, including input-timeout rejections; it does not limit configured execution time. This keeps HTTP/2 streams open during admission and longer evaluations rather than resetting them before their terminal envelope can be written. If the delivery deadline cannot be installed, the server reports the failure and aborts the connection/stream without attempting headers, an envelope or a fallback; no terminal result is available over that failed transport. CLI signal cancellation does not require the stdin producer to close its pipe. Execution observes request/eval/task deadlines and performs synchronous cleanup. Host-owned factories must return promptly: arbitrary host callback code is not sandboxed or forcibly preempted.

One validated envelope receives one output attempt. Short writes, write errors and HTTP flush errors are never followed by a second envelope or fallback. CLI output failure exits 2 even if partial bytes already contain another encoded exit. HTTP write/flush failures are logged; a peer may see no response or partial bytes. A successful write is not downstream receipt, durability or atomic publication. HTTP runner error reporting is nonblocking: at most one diagnostic reporter may wait on the logging sink. Additional reports are counted in suppressed_diagnostics on the next accepted report, not queued as more blocked goroutines. A permanently blocked sink cannot guarantee diagnostic delivery; transport abort, handler return and cancellation completion do not wait for it. The Lab validates the captured request with a separate ten-second client limit. After validation, its terminal-response limit uses the captured timeout_ms plus fifteen seconds for body admission, runtime cleanup and delivery (five seconds each). Rejected requests receive only that fifteen-second allowance. Editing the request or leaving the view cancels the active request. SIGKILL, process crashes and host loss cannot guarantee a terminal record. There is no scheduler retry, cross-worker deduplication or exactly-once execution. Unix signal tests do not establish native Windows signal coverage. The Unix CLI bounds its sole inherited stdout write attempt to five seconds without changing shared open-file-description flags. Retained stdin/stdout peers keep their status flags, including legitimate flag changes made by peers during I/O; cleanup does not restore stale flags over them. This is a process-exit bound, not a deadline guarantee for arbitrary embedding writers. The Windows CLI likewise bounds one inherited-stdout emission attempt to five seconds, including pipes without write-deadline support, without changing shared handle modes/deadlines or closing stdout. Completed emission preserves the selected terminal outcome; native write failure, short write or an exhausted output budget selects transport exit 2. A partial record or blocked write may remain until process exit; this does not prove receipt or preempt arbitrary embedding writers. Before exit, Cobra and the final CLI error diagnostic share a separate 100 ms cumulative stderr write-wait budget; time between writes does not consume it. Writable stderr retains the same message bytes. Blocked, failed or short diagnostic delivery can be absent or partial; it stops further diagnostic attempts but never changes the selected exit code or adds a stdout terminal record. A full retained stderr pipe therefore adds at most this budget after the five-second stdout decision, not an unbounded exit delay. The process-only policy does not close inherited stderr, change shared flags, or bound unrelated command logging or arbitrary embedding writers. Supplied-input and inherited-Windows interruption errors return through the normal joined-error path rather than printing synchronously from cancellation callbacks. Callback completion and deadline-reset errors remain joined; native interruption operations themselves must still return. A request rejected before HTTP body admission is not drained before its terminal response. Its HTTP/1.1 connection closes even if the client sent the full body; clients must reconnect rather than relying on keep-alive for these rejections. Fully read requests retain keep-alive, including admitted terminal failures.

Lab and export

In live Runner contract, use Run selected unit explicitly. Compatibility and Validate selection remain offline. The result shows the producer outcome, exit, boundary, validation, observed execution mode, unavailable billing and linked diagnostics. The browser validates the schema and any claimed input identity binding; the producer checks the complete grader binding. Export terminal JSON saves the actual envelope. Editing the request or starting a new run removes old success; response errors do not leave a stale passing result. There is no execution in the synthetic demo. An HTTP rejection before run admission remains visible without claiming an execution identity, even when the separate offline validation succeeded.

Actual mock producer terminal evaluation failure, exit 1, failed observed expectation and linked diagnostics in Lab.

🔍 Click to enlarge

Migration and extension rules

Earlier unshipped source-preview runner run responses used waza.runner-unit. Switch consumers of run only to waza.runner-terminal: use selection instead of resolved, outcome/exit_code instead of receipt status, and the bounded observation instead of a full RunResult. Cancellation and timeout now have exits 3 and 4 rather than generic execution exit 2. Invalid input/flags have exit 6. Validate and negotiate keep their existing non-execution receipts.

Historical local waza run, eval/task schemas, saved evaluation artifacts and identity algorithm are unchanged. Do not rewrite old receipts into terminal success or infer missing identities/billing. Incompatible terminal fields, meaning or exit mappings require a new terminal schema version. Native, archive and container capability names require their own implementation and review. This page is contract documentation, not a release announcement or release blog.