Runner contract and capability negotiation

Source-build preview for v0.40.0. The implemented capabilities are negotiation, exact-unit, inline.mock-text, terminal-result and inline.copilot-text. These bounded profiles execute one captured text attempt. The factory result must match the selected profile: a non-nil concrete mock engine or a privately owned, diagnostic Copilot engine; other implementations and wrappers fail before any engine lifecycle call, with zero calls, no observation and no claimed execution mode. Admitted receipts carry input-derived identities. Copilot execution requires explicit paid consent and disables all tools and skills. General native file/tool/skill workflows and archives remain unavailable. A local container reference exercises the mock/text profile; no release or container publication is announced here. The local compatibility adapter captures a narrower one-task mock/text suite offline; explicit run --runner-compat uses the same execution service while retaining historical artifacts and local exits.

CLI: JSON stdin and stdout

Build this source with make build, then inspect its capabilities:

waza runner capabilities

{"schema_version":"1.0","versions":["1.0"],"capabilities":["negotiation","exact-unit","inline.mock-text","terminal-result","inline.copilot-text"]}

Negotiate without launching a model, tool, subprocess or runtime:

waza runner negotiate < examples/runner/negotiation.json

The example offers a future version with an explicit 1.0 fallback:

{
  "schema_version": "1.0",
  "versions": ["2.0", "1.0"],
  "required_capabilities": ["negotiation"],
  "optional_capabilities": ["exact-unit"]
}

The response has kind waza.runner-negotiation, schema version 1.0, status compatible, and a negotiation object. Its selected version is 1.0, selected_capabilities is ["exact-unit","negotiation"], and unavailable_optional_capabilities is []. No execution occurred. Requested versions, required/optional capabilities and available capabilities are echoed separately. Empty arrays stay arrays, not null.

Requiring "inline.native-text" instead returns unsupported_capability with an actionable diagnostic and exit 5, before interpreting any input or constructing an engine. Names are exact and case-sensitive: Negotiation, negotiation.extra and negotiation do not silently become negotiation.

Status	CLI exit	Meaning
compatible	0	All required capabilities are implemented for the selected version; not execution success
unsupported_version	5	No mutual version; offer 1.0 explicitly
unsupported_capability	5	A required capability is unavailable
invalid_request	6	Invalid JSON/schema, conflicting required/optional names, interrupted input, or command misuse

Failure to write the response exits 2. Standard output contains just one JSON response; command errors use standard error. Update discovery is disabled for all runner commands. SIGINT/SIGTERM while waiting for stdin can reject admission without waiting for a producer to close its pipe. Uncatchable termination, broken output pipes, and process crashes cannot guarantee response delivery.

Execute one exact work unit

The complete example contains one captured eval and task, not paths to read:

waza runner validate < examples/runner/unit.json
waza runner run < examples/runner/unit.json

validate performs the same negotiation and admission as run, but creates no engine. run explicitly starts one fresh profile-matching runtime and executes exactly one attempt. The example selects:

{"shard":"shard-a","task":"selected","trial":2,"attempt":2}

Trial and attempt are one-based selectors, not counts of work to execute. The example’s eval declares three trials and three maximum attempts. Selecting 2/2 does not run trial 1, attempt 1, remaining retries, or adjacent tasks. The captured eval’s tasks patterns are never expanded or opened: the supplied input.task is the only declared candidate. input.shard and the task’s id must exactly match the selection. Names, file patterns and display names are not interchangeable selectors.

Integer fields follow JSON Schema numeric semantics, not just integer spellings: "timeout_ms": 6e4, "trial": 2.0, and "attempt": 20e-1 mean 60000, 2, and 2 in negotiation, validation and execution. Conversion is exact, without floating-point rounding. Fractional values, out-of-range values, nulls and non-number types are rejected; the existing schema and declared eval bounds still apply.

The supported inline.mock-text profile requires both exact-unit and inline.mock-text in required capabilities, executor: mock, a captured inline inputs.prompt, disabled ambient skills, and at least one valid builtin text grader or nonblank text expectation. The example uses a skill-free 2.0 scenario; task skill_directories: [] or explicit deny-all skills are also supported where the shared effective-skill rules disable discovery. The inline prompt must be explicitly present: inputs: {prompt: ""} is valid, but omitted inputs, inputs: {}, or a prompt-file-only task is rejected before runtime construction. Empty strings are not treated as missing prompts. Authored skill paths, external files, prompt files, context, environment variables, workdirs, repositories, MCP/command mocks, hooks, parallel execution, baselines, CSV sources, responders, follow-ups and checkpoints are rejected, not silently approximated. Captured YAML admits one document with exact native field names and scalar types. Aliases, anchors, merge keys, duplicate/non-string mapping keys, unknown fields, authored floating-point values and wrong scalar types (including null for non-nullable scalars) are rejected. Native nullable pointers and collections retain their existing null handling; original source strings remain unchanged. Invalid grader parameters, disabled tasks, conflicting IDs and out-of-range selectors also reject before runtime construction. Execution counts and timeouts must fit nonnegative signed 32-bit integers; the existing native loader additionally requires positive trial and timeout values. Omitted max_attempts or explicit zero means one allowed attempt, not an automatic retry. A task-level zero first-event timeout explicitly disables that watchdog; absent task timeout inherits the eval value.

Only the selected attempt’s engine response is graded. Cache replay and automatic retries are absent. An explicit repeat starts a fresh engine and workspace and executes the same tuple again; this is not cross-worker exactly-once execution, deduplication or idempotent side effects. Cancellation and the request/eval/task deadlines bound the attempt, and shutdown is attempted with a separate five-second budget. HTTP server shutdown cancels active requests and waits for handler cleanup.

Both exact-unit profiles accept eval metrics as uncomputed, identity-bound metadata. A successful terminal result reports only the selected attempt: it does not report aggregate metric values, metric-threshold decisions or whole-evaluation success. Nonempty declarations are validated offline against the canonical eval schema’s metric definition using original numeric values; malformed, wrong-type and out-of-range nonempty declarations reject before runtime construction. For historical compatibility, explicit metrics: null decodes as nil, like an omitted field, in the shared captured reader used by mock execution and native snapshot reconstruction; exact-unit Copilot admission still rejects explicit null. Existing empty arrays also remain accepted. Numeric values must be finite and must not underflow the native numeric range. The local compatibility adapter independently rejects an explicit metrics field, including null, rather than dropping unsupported metadata. This captured-reader compatibility does not relax strict canonical eval-schema admission, which still requires a nonempty metrics array. The public Copilot example includes a nonempty schema-valid declaration; changing it changes identity, not the attempt-only execution scope. The local compatibility adapter still rejects nonempty metrics rather than projecting aggregate claims.

Offline validation returns waza.runner-unit, an invocation-local selection receipt. Execution now returns waza.runner-terminal, with a bounded observed summary and stable exit category. Internal unit receipt fields remain:

Field	Interpretation
compatibility	Shared negotiation response, separate from selection validation
requested / resolved	Complete tuple; resolved is absent on rejection
validation	not_checked, invalid, or valid
adjacent_work_selected	Always false; no adjacent candidates are loaded
engine_calls	Observed actual engine invocations, zero or one
status	ready after validation only; rejected, completed, or execution_error
execution_mode / observation	mock or copilot plus the original attempt’s RunResult when available
identity	Optional versioned input digest and logical hierarchy, derived after admission and before runtime construction

completed means a graded attempt completed, not that every check passed. Inspect observation.status and validations. Logical IDs describe admitted input, not executed work. No publication, billing estimate or archive assurance is invented. The nested observation remains the existing attempt shape; this layer provides no receipt ingestion/archive reader.

Run exits are 0 success, 1 failed checks, 2 infrastructure/output failure, 3 cancellation, 4 timeout, 5 unsupported negotiated requirements and 6 invalid input. These are runner-local categories; legacy commands remain unchanged. Broken output and uncatchable termination cannot guarantee delivery. Paid consent alone never enables an unadvertised or optional profile.

Opt-in production Copilot text execution

Require exact-unit and inline.copilot-text (not inline.mock-text), set config.executor: copilot-sdk, choose an explicit nonblank config.model, and set allow_paid_execution: true. The captured example uses the same 2/2 tuple:

# Offline: no authentication, subprocess or inference.
waza runner validate < examples/runner/copilot-unit.json
# Opt-in inference: requires your configured Copilot runtime and authentication.
# This may incur charges on every dispatch, including explicit repeats.
waza runner run < examples/runner/copilot-unit.json

The profile keeps the mock profile’s input restrictions and builtin text checks. The SDK receives skills disabled, an empty tool allowlist, deny-all permission checks and a denying pre-tool hook. Runtime configuration discovery, repository-origin file hooks and on-demand instruction discovery are explicitly disabled in the SDK session settings, not left at runtime defaults. No explicit hook/plugin/instruction roots are supplied. Ordinary execution defaults are unchanged. Custom instructions and embedding retrieval are skipped, shared session-store access and memory are disabled, and host git context is disabled. The eval executor is copilot-sdk; the terminal mode is separately copilot. The post-create/resume session.options.update also carries skipCustomInstructions: true; rejection fails before any prompt is sent. SDK capture/wire controls verify the requested settings; they do not attest runtime discovery behavior or a host filesystem sandbox. It cannot consume fixtures, files, MCP, agent definitions, responders or follow-ups. A tool attempt is an execution failure, not permission to broaden the request.

Each invocation owns its SDK client and workspace. Request cancellation and the task/eval timeout force-stop that private transport; shutdown has a separate five-second budget and also interrupts the transport on expiry. Cleanup and incomplete-usage diagnostics fail explicitly. The terminal reports the observed copilot mode but does not publish billing or attest provider provenance. Credentials and supported provider/runtime environment settings remain deployment inputs, not identity-bound content. Validation proves no runtime readiness. An effective nonzero first_event_timeout_seconds is rejected before runtime construction because its legacy child-context watchdog cannot interrupt an uncooperative RPC. Use task/eval timeout_seconds or request timeout_ms. A task override of zero can disable the eval-level first-event setting.

Deterministic tests use SDK fakes, including a send that ignores cancellation until its transport is killed. They do not prove live-provider behavior or native-platform coverage. Additional maintainer controls build the production CLI and run it against a synthetic stdio subprocess with no inherited credentials/provider configuration. They validate terminal JSON, identities, the exact selected tuple, grader verdicts and the full RPC inventory for success, failed checks, session error, timeout, startup failure, protocol mismatch and unsupported capability. See the offline production-process controls for exact commands and limitations. Synthetic responses are not bundled-runtime, inference, billing, sandbox or container-image assurance. No termination guarantee covers a custom transport whose force-stop operation never returns, arbitrary filesystem hangs or host loss. The identity algorithm and historical mock IDs do not change: the authored executor/model already participate in the canonical eval digest. General waza run and its mock-only compatibility adapter are unchanged.

Troubleshooting selection

For selection conflicts with declared input shard/task, compare shard labels and the captured task’s exact id; do not pass a filename/glob as selection.task. For an out-of-range trial/attempt, declare that ordinal in the captured eval or select an existing ordinal. For an unsupported profile, retain waza run rather than dropping semantics to make admission succeed. Never treat compatibility alone or a declared tuple as evidence of actual execution.

HTTP API

waza serve --lab --no-browser exposes the same offline contract:

Method and route	Behavior
GET /api/v1/runner/capabilities	Same advertisement as runner capabilities
POST /api/v1/runner/negotiate	Same request and response as runner negotiate
POST /api/v1/runner/validate	Offline exact selection validation
POST /api/v1/runner/run	Explicitly execute one admitted mock/text or opted-in Copilot/text attempt

curl --silent --show-error \
  -H 'Content-Type: application/json' \
  --data-binary @examples/runner/negotiation.json \
  http://127.0.0.1:3000/api/v1/runner/negotiate

Use --data-binary @examples/runner/unit.json and the /validate or /run route for the same complete request used by the CLI.

A complete admitted JSON request returns HTTP 200; inspect negotiation/validation status or run outcome/exit_code rather than treating transport success as compatibility. Missing/non-JSON media type returns 415. Negotiation/validation oversized or failed uploads return 400. Run returns terminal bodies for invalid media (415), oversize (413), read failure (400), interrupted/timed-out body (408) and infrastructure failure (500), when output is writable. A five-second body read deadline is installed before media-type validation, bounding partial uploads even when the media type is missing or invalid. Negotiation/validation may drain unread request bytes before completing a 415. Run rejects without draining an unadmitted body: its HTTP/1.1 connection closes even if the client sent the complete body, so the client must reconnect. Incomplete timed-out uploads also close their connection. Completed negotiation/validation requests and fully read run requests retain keep-alive, including admitted terminal failures. Middleware must preserve response-controller read-deadline support; inability to install the deadline fails closed with HTTP 500. Transport failures are not evaluation verdicts. The API reads no host input paths and stores no runner receipts. It retains the existing server; no cloud authentication, scheduler or worker registry is added.

Paid native HTTP execution is local-only. The immediate peer must be a loopback IP and Host must be localhost or a literal loopback IP. A supplied browser Origin must exactly match the request’s HTTP(S) scheme and authority; cross-site and same-site (not same-origin) Fetch Metadata are rejected. Forwarded headers are not trusted. Rejected requests return HTTP 403 and a native_http_denied terminal diagnostic with zero engine calls and no execution identity or observation, before runtime construction. DNS Host aliases are not resolved or admitted, including aliases that resolve to loopback.

Offline validation/negotiation, remote mock execution and existing remote read/static serving retain their previous behavior. Use local waza runner run for native execution outside this HTTP boundary. This policy trusts local processes, not individual OS users, and adds no remote-user authentication. Do not expose the paid endpoint through a reverse proxy that rewrites remote peer, Host or Origin into trusted local values. allow_paid_execution is request consent, not server-owner authorization.

If final usage is unavailable after disconnect, isolated runner sessions do not resume solely to retrieve shutdown usage, because that SDK fallback uses unrestricted defaults. Missing usage remains explicit and fails terminalization, not recorded zero. Ordinary execution retains its existing fallback.

Schemas and presence

• Negotiation request 1.0 schema
• Negotiation response 1.0 schema
• Complete runner request 1.0 schema
• Capability advertisement 1.0 schema
• Exact unit receipt 1.0 schema
• Terminal envelope 1.0 schema

All four compact request fields are required. versions must contain at least one distinct major.minor string. Capability arrays may explicitly be empty; omitted, null, false, zero or empty-string arrays are invalid. Duplicate JSON keys, unknown fields, duplicate array entries, malformed Unicode and trailing JSON are rejected. Compact arrays are limited to 64 entries. The wire size limit is 4 MiB. Invalid-request messages are bounded to 4,096 Unicode characters. Large complete requests report at most 64 individual missing required capabilities plus an explicit remaining count; the full requested lists remain available.

The API also accepts the complete captured runner request defined by the separate schema. Its fields are schema_version, versions, required_capabilities, optional_capabilities, run_key, selection, input, timeout_ms and allow_paid_execution. All are required. Explicit allow_paid_execution: false is valid; omission or null is not. Selection requires shard, task, positive trial and positive attempt. Input carries inline shard/eval/task strings, not host paths. negotiate only shape-checks this request: captured YAML, semantic selection validity and payment permission are never executed or interpreted by negotiation. validate/run additionally perform the bounded profile’s semantic admission; only run starts the admitted profile’s runtime. Negotiation alone never enables paid execution or emits terminal results. Do not treat compatibility as work admission. Any complete-only field selects this schema, so partial complete requests report their missing or invalid fields rather than compact-profile unknown fields. Shape checks accept equivalent integer JSON numbers, for example "timeout_ms": 6e4 and "selection": {"shard": "s", "task": "t", "trial": 2.0, "attempt": 1}. These numbers are validated against the published bounds without converting execution values into runtime settings.

Compatibility, migration and evolution

Negotiation envelope schema version 1.0 is separate from the offered runner versions. Unsupported envelope versions are invalid_request; unsupported runner offers are unsupported_version. Version offers are exact strings, not ranges. 01.0 is not 1.0. With several mutual versions, the server chooses its highest supported version, not client order. This implementation supports only 1.0.

Required and optional names must not overlap. Unsupported optional capabilities are listed without rejecting the request; unsupported required names reject the whole request and leave selected capabilities empty. No missing capability is silently downgraded. New compatible implementations may add advertised names; breaking semantics require a new version or capability name. Do not infer features from the Waza binary’s release number.

Existing waza run, eval/task schemas and historical results are unchanged. No published local command is deprecated by this layer. Migrate external compatibility checks to the explicit runner namespace, retain your current execution command for unsupported profiles. To use the new exact path, capture one task explicitly, require its profile, and submit a complete request to runner run. Do not replace local multi-task commands or relabel historical artifacts as exact-unit receipts.

The negotiation response has no execution outcome, logical identity, scores, billing or terminal result. It must not be ingested as an evaluation result. See the developer boundary and layer allocation and runner release roadmap.

Dashboard Lab

Open waza serve --lab, then Runner contract. Edit the JSON request and select Check compatibility. The live view displays the negotiated API/runner version, requested/available/selected capabilities, unavailable optional names and incompatibility diagnostics from the public API. The check never runs an evaluation. Editing or retrying clears prior results; failed refreshes show an error instead of stale compatibility.

🔍 Click to enlarge

The Validate exact work selection panel sends the complete captured example to /validate. It displays requested/resolved shard/task/trial/attempt, validation state, no adjacent work selected, and zero engine calls. It does not call /run. Editing or failed reload clears old selection state. The same API result displays the derivation version, canonical input digest and full run/shard/task/trial/attempt identity hierarchy, with Export validation JSON. This is input-derived metadata, not execution/provenance evidence. Rejected or older receipts show identity as unavailable; no ID is fabricated.

🔍 Click to enlarge

The synthetic demo does not call the runner API or fabricate negotiation data. On an older server or static host, the check reports an error, not success.

The separate Run selected unit action actually executes the captured request, then shows the terminal outcome, exit, diagnostics and JSON export.

Initial contract notes (unreleased)

This adds negotiation and exact inline text execution without changing local result or eval versions. No older binary gains these commands automatically. There is no automatic paid-provider call, container digest, durable publication or strong-isolation guarantee in this layer. Existing mock requests need no migration; opt-in Copilot requests must require the new capability and consent. An older server rejects that capability before execution. The v0.40.0 release article is deferred until the release’s required items are merged.