# W2 / releases#238 — mid-turn agent output: DESIGN PROPOSAL **Author:** todlando · **Date:** 2026-08-29 · **Gate:** doyle, against `SEMAPHORE-242-GRILL.md` **Measurement sha:** `786d2381` (= `main` = `origin/main` at the time of writing; verified with `git rev-parse HEAD` in `.worktrees/ir57` and `git log --oneline origin/main -1`). Every file:line below is at that sha. **Nothing is built yet** — this document is the artifact to gate. Operator ruling under gate: ATTRIBUTE on `AGENT_OUTPUT` (releases#238 comment 5461768445), and core-owned shortform **and** `;;` seal markers parse ALL agent output including mid-turn. --- ## 1. The attribute's shape, and the #234 reader's tolerance — MEASURED, and the precedent does NOT transfer ### 1.1 What was verified The grill said: *verify the shipped #234 reader's tolerance of an unknown attribute (attr-passthrough precedent) at design.* Verified at `786d2381`. **The two layers have opposite postures, and only one of them is the #234 reader.** **The wire/EVENT layer DOES pass unknown attributes through** — this is the precedent, and it is real: - `docs-site/src/messaging/overview.md:193-194` — `[doc->REQ-EVENT-ATTR-PASSTHROUGH]`, *"The attribute set is open — re-render by pass-through, never by allowlist."* - `crates/spt-proto/src/event.rs:255` — `attr_key_is_reemittable`: an unknown attribute IS re-emitted, and only a key outside `[A-Za-z0-9_-]` is dropped (fail-closed on charset, not on vocabulary). **The #234 reader does NOT.** It is closed by construction, at two sites: 1. `crates/spt-store/src/iolog.rs:98-119` — `struct IoLogRow`: six named fields (`at_ms`, `kind`, `payload`, `truncated`, `digest_seq`, `peer`, plus the `serde(skip)` line-key `seq`), **no `#[serde(flatten)]` catch-all and no `deny_unknown_fields`**. serde's default therefore applies: an unknown JSON key in a stored row is **accepted and DISCARDED** — the row survives the parse (`parse_line`, `crates/spt-store/src/iolog.rs:148-155`) and the value is retained nowhere. 2. `crates/spt/src/api/ioevents.rs:237-262` — `render_json`: an **explicit per-field allowlist projection** into a fresh `serde_json::Map` (`seq`, `at_ms`, `kind`, `payload`, then `truncated` / `digest_seq` / `peer` only when set). There is no generic path; even a retained unknown field could not reach the adapter through this function. **Verdict: an unknown attribute is IGNORED — not passed through, and not refused.** The attr-passthrough precedent is a fact about the EVENT wire, not about this reader, exactly as the grill's "verify, do not assume" anticipated. ### 1.2 Two adjacent facts, both measured, that shape the design - **The ignore-not-refuse posture that IS tested covers the `kind` VALUE, not an unknown FIELD.** `crates/spt/src/api/ioevents.rs:173-177` documents it and `:416` (`an_unknown_kind_is_ignored_not_refused`) pins it. Unknown-FIELD tolerance is true only by serde's default — **decided by nobody, pinned by no test.** *(That tolerance is now MEASURED, not inferred from the absence of `deny_unknown_fields`: a probe unit `probe_a_row_with_an_unknown_field_still_parses` feeding `parse_line` a row carrying `"mid":true` passes — `cargo test -p spt-store --lib iolog::tests::probe`, 1 passed / 0 failed, exit file 0, in `.worktrees/ir57` at this sha plus that one test. The probe is an untracked lane edit; §6 proposes keeping it as the lane's forward-compat unit.)* A later "hardening" edit adding `deny_unknown_fields` would make `parse_line` return `None` for every row a newer core wrote, and `read_after_at` (`crates/spt-store/src/iolog.rs:318`) **skips a `None` row silently** — the poll would not refuse, it would go quiet about real events. That is the forward-compat hazard this lane should close with a unit, whatever else it does. - **The store keeps what the reader drops.** `trim_locked` (`crates/spt-store/src/iolog.rs:268-285`) filters whole LINES and rewrites them verbatim; it does not round-trip rows through `IoLogRow`. So an unknown key survives on disk through a trim. The loss is at the reader, and only at the reader. ### 1.3 What an old reader silently drops, and why that is acceptable An N-1 `spt` polling a log written by an N core sees mid-turn `AGENT_OUTPUT` rows **as ordinary `AGENT_OUTPUT` rows with the `mid` field absent** — i.e. it reads them as end-of-turn reports. It does not refuse, does not skip them, and does not stall its cursor. The degradation is: an old consumer over-counts turns. **This is acceptable and is NOT version-gated, for three reasons:** 1. Nothing emits a mid-turn span unless an adapter asks for one (§2), and an adapter that asks has declared IO compliance in its own manifest — the same handshake `REQ-IO-SHORTFORM-GATE` already uses to keep a core upgrade from changing behaviour under a shipped adapter. 2. The alternative — a new `kind` — is exactly what the operator ruled AGAINST, on the semantics ("it'd be confusing if `AGENT_OUTPUT` excluded *some* agent output"). A version gate is that same refusal wearing a flag. 3. The mis-read is *over*-counting, never a wrong payload: the body an old reader gets is real agent output, verbatim, in order. **What this does obligate:** the attribute must be minted as a NAMED field at every site, because "it rides through as an unknown attribute" is false here. Five sites, and the design carries all five or the attribute is invisible: | # | site (at `786d2381`) | what it gains | |---|---|---| | 1 | `crates/spt/src/api/mod.rs:121-136` (`ApiCmd::State`) | `--mid` flag | | 2 | `crates/spt/src/api/delivery.rs:85-95` (`state_io_kind`) | a third argument (§1.5) | | 3 | `crates/spt-daemon/src/iobus.rs:40-58` (`IoEvent`) | `pub mid: bool` | | 4 | `crates/spt-daemon/src/shellchan.rs:175-190` (`compose_io_frame`) | `mid="1"`, present-only | | 5 | `crates/spt-store/src/iolog.rs:98-119` + `crates/spt/src/api/ioevents.rs:237-262` | the row field + its projection line | ### 1.4 The attribute: `mid`, present-only **`mid="1"`, present-only, exactly the house form `truncated` already has** (`crates/spt-daemon/src/shellchan.rs:187-188`; row field `#[serde(default, skip_serializing_if = "std::ops::Not::not")]`, `crates/spt-store/src/iolog.rs:110`; JSON `"mid": true` emitted only when set). ```text payload ``` Rejected: `span="mid"|"final"`. An enum invites an ordinal ("first/…/last"), and **core cannot know a span's ordinal without modelling turn assembly** — the thing §2 forbids. A present-only marker says precisely what the adapter knows and core can carry: *this payload is a mid-turn span, not the turn's closing report.* Absence keeps its shipped meaning, which is what makes the field additive. ### 1.5 How an adapter reports one — and the one refusal this adds A mid-turn span is reported while the agent is BUSY, so it arrives on the `busy` arm; but its kind must be `AGENT_OUTPUT`, not `USER_INPUT`. `state_io_kind` gains a third **argument** (not ambient state — its purity comment at `crates/spt/src/api/delivery.rs:77-84` is load-bearing and survives): | state | payload | `--mid` | kind | |---|---|---|---| | `busy` | yes | no | `USER_INPUT` (unchanged) | | `busy` | yes | **yes** | **`AGENT_OUTPUT`, `mid` set** | | `idle` | yes | no | `AGENT_OUTPUT` (unchanged) | | `idle` | yes | **yes** | **refused by name — `STATE_MID_ON_IDLE`** | | either | no | either | `None` — no event (unchanged back-compat arm) | `--mid` at `idle` is a contradiction (a mid-turn span at turn close), and `--mid` with no payload asks to mark nothing. Both are usage bugs and are **refused by name**, at the same place and for the same reason `STATE_PAYLOAD_AMBIGUOUS` is (`resolve_state_payload`, refused before the auth gate so an ambiguous call is not reported as unauthorized). This is argument validation, not a funnel outcome — the funnel's "an observation can never fail the operation it observes" (`crates/spt/src/api/delivery.rs:97-103`) is untouched. --- ## 2. Mid-turn parse under the exactly-once span respec — ruled, restated, and it costs NO new parse site Restating the ruling (grill, #238 non-operator half) so the build carries it: **non-overlap stays the ADAPTER'S reporting contract** — "every reported payload span is reported EXACTLY ONCE across the turn" (mid-turn chunks + closing remainder disjoint) — and **core's parse stays per-ingest-call with no cross-call dedup machinery**, because core-side span dedup would require core to model turn assembly, which the funnel design refuses on purpose. Measured consequence, and it is the pleasant one: **the ruled scope extension needs no new parse site.** `cmd_state`'s `busy` payload arm already calls `dispatch_ingest_shortform` and `dispatch_ingest_seal_mints` under the manifest gate (`crates/spt/src/api/delivery.rs:349-356`), and the `idle` arm does the same (`:328-335`). A mid-turn span arrives on the `busy` arm, so **it is already parsed the moment the kind is right.** W2's parse work is therefore: get the kind right (§1.5), decide the bare-marker case (§3), and document the contract (§4) — not plumb a parser. The ingest-scope boundary is unaffected and stays structural: `dispatch_ingest_shortform` is reachable only from `cmd_state`'s payload arms and never from the inbound path (`crates/spt/src/api/delivery.rs:123-134`, `REQ-IO-SHORTFORM-DISPATCH`). A mid-turn span is one more call on an edge that was already inside the boundary. **What an adapter that breaks the contract gets:** overlapping spans dispatch a tag twice — two messages, not a refusal. Named in the docs (§4) so an author can recognize the symptom, because core cannot detect it without the state it refuses to keep. --- ## 3. The bare `;;` under chunked ingest — THE DECISION ### 3.1 The problem, exactly `crates/spt-proto/src/shortform.rs:243-277` (`parse_seal_mints`): pairs parse greedily left to right (ruling 12), an empty pair is fully ignored (ruling 13), and **an odd trailing marker is the bare case: it seals everything after it THROUGH END OF OUTPUT** (ruling 14, `:266-276`). "Through end of output" is a claim about text the parser **has not seen** when a mid-turn span is parsed. Parse a bare marker per-chunk and it seals to the end of *that chunk* — a strictly smaller region than the author asked for, and **silently**, because a short seal is indistinguishable from a correct one in the output. A seal is a human-presence ceremony over a TEXT; sealing the wrong, shorter text is a correctness failure wearing success's clothes. ### 3.2 The options, priced - **(A) Defer to turn close with per-turn parser state** *(the gater's lean).* To honour ruling 14 the sealed region runs from the marker to end of turn — text spread across calls core did not retain. So core must **buffer the remainder of the turn's output, per endpoint**, and know when a turn ends. That is turn assembly by any name; it is unbounded in row count on the hot path (the 16KB cap bounds a frame, not the count); and it leaks, or seals the wrong region later, when a turn never closes (crash, no `idle` call). - **(A′) Defer only the MARKER and seal the closing payload at turn close.** Cheap, and **wrong**: it seals the closing span, not from the marker onward. Silently wrong region again — worse than (B), because it looks deliberate. - **(B) Parse per-chunk; the bare marker seals to end of chunk.** No new state, silently under-seals. Rejected: this is precisely the failure §3.1 names. - **(C) A bare marker in a MID-TURN span is refused by name; the bare case is reachable only at the turn-close ingest.** Pairs — self-delimiting, both ends present in the chunk — keep parsing mid-turn exactly as ruled. ### 3.3 DECISION: (C) **A `;;` PAIR parses mid-turn and mints as ruled. An odd trailing `;;` in a `mid` span mints nothing and is refused by name (`SEAL_BARE_MIDTURN`), routed into the shipped `DISPATCH_RESULTS` surface — no rival refusal beside it, the same routing `SEAL_NO_CEREMONY_SURFACE` already takes (`REQ-IO-SEAL-SHORTFORM-CEREMONY`). At the turn-close ingest (`idle`) the bare case is UNTOUCHED: "through end of output" is text the parser holds in full, and ruling 14 stands verbatim there.** Grounding: 1. **It is the only option where what gets sealed is a text the parser has seen in full.** (A) achieves that too, but only by buying the buffer; (A′) and (B) both seal a region the author did not name. 2. **The bare marker is by construction a turn-close gesture, so mid-turn handling buys nothing.** The point of mid-turn parse is latency — a tag fires when the agent writes it instead of at turn end. A pair gets that in full. A bare marker's region *does not exist yet* when the marker is written, so there is no early work to do and no latency to win. Refusing it mid-turn costs the author nothing they could have had. 3. **It costs core zero cross-call state, so §2's no-turn-assembly holds without an exception.** Under (A), §2's rule survives only as "no turn assembly except for seals", and an exception in a boundary rule is how the boundary stops being one. 4. **The failure is LOUD.** The author learns the gesture's scope instead of receiving a shorter seal that looks correct. This does **not** contradict ruling 13's silence for `;;;;`: there the author asked for nothing, so a refusal is noise about a non-event; here the author asked for something real that cannot be honoured, so silence is the noise. 5. **The `mid` attribute is what makes this decidable with no state at all.** Core does not need to know where it is in a turn — it needs to know *whether this payload is the turn's closing text*, and §1's attribute is exactly that observable, carried by the party that knows (the adapter). Point 1 and point 3 are the same mechanism used twice. The refusal names the fix: *close the pair, or place the bare marker in the turn's closing output.* **Deviation declared:** this is narrower than the gater's lean (A). It honours the operator ruling — seal markers DO parse mid-turn — and carves out the one gesture that cannot be honoured mid-turn without buffering. If the gater prefers (A), the price is the buffer, the leak-on-crash arm, and the exception in §2; say so and I will build it with a bounded buffer and a named drop when either the bound or the turn is exceeded. --- ## 4. Doc sweep + the rate re-derivation — enumerated by grep, not recall All sites at `786d2381`: | # | site | change | |---|---|---| | 1 | `docs-site/src/shells/frames.md:294` | taxonomy row `AGENT_OUTPUT \| the agent finishes a turn` → the agent produces output (a mid-turn span, or the turn's close) | | 2 | `docs-site/src/shells/frames.md:384-389` | **"`AGENT_OUTPUT` reports the END of a turn, not the whole of it … Mid-turn output belongs to your adapter's `[digest]` feed"** — now false; rewritten as the mid-turn paragraph | | 3 | `docs-site/src/shells/frames.md:271` + `:277-285` | frame example gains `mid="1"`; the attribute list gains the `mid` bullet with its present-only reading | | 4 | `docs-site/src/shells/frames.md:379-383` | "**send one payload-carrying report per turn**" → the exactly-once-span contract (§2), with the overlap symptom named | | 5 | `docs-site/src/shells/frames.md:312` | the parse-vs-cap ORDERING sentence (queued from perri's filing): parse runs over the FULL ingested payload; the 16KB cap bounds only the EMITTED frame body | | 6 | `docs-site/src/shells/frames.md:391-395` | "Report a turn's output once" — respec'd to spans | | 7 | `docs-site/src/harness-contract/patterns.md:132` | dedup note → exactly-once spans | | 8 | `docs-site/src/harness-contract/integration-checklist.md:97` | `api state` row: "Report **one payload-carrying call per turn**" → exactly-once spans; `--mid` named | | 9 | `docs-site/src/harness-contract/integration-checklist.md:99` and `:362` | `api io-events` row + checklist item: the `mid` field in the poll's JSON | | 10 | `docs-site/src/harness-contract/api.md` | `api state` verb (the `--mid` flag + `STATE_MID_ON_IDLE`) and the `api io-events` output field list | | 11 | `CONTEXT.md` (funnel entry, touched by #234) | the taxonomy sentence, same amendment as row 1 | | 12 | `crates/spt-proto/src/shortform.rs:233-241` + `crates/spt-proto/src/ioevent.rs:1-13` | the bare-case scope (§3), stated where the grammar lives | ADR-0048 decisions 1-3 stay perri's locked contract — restated, not reopened. ### 4.1 The rate re-derivation (issue item 4) The figure and its premise live at `crates/spt-store/src/iolog.rs:47-83`. **Reproduced as a leg, not quoted:** premise = TURN-BOUNDARY emission, two events per boundary; measured peak 43 boundaries/hr over 1117 harness transcripts (2026-08-28) ⇒ ~130 events/hr; `1000 / 130 = 7.69` ⇒ the module's "~7.5 hours of the worst hour ever measured" reproduces. Typical busy hour 13-16 boundaries ⇒ ~45/hr ⇒ ~22h, the module's "roughly a day of ordinary busy work". **The module already carries this lane's number and names this lane as its owner** (`crates/spt-store/src/iolog.rs:76-82`): a mid-turn emitter at the digest-entry quantum puts the peak near **~540/hr on the same corpus**, "deliberately NOT folded in here … belongs to whichever request lands that emitter". That is us. Re-derived: - peak horizon: `1000 / 540` = **~1.85 h** (was ~7.7 h) - ordinary busy horizon: `540 / 130 ≈ 4.15×` the old rate ⇒ ~187/hr ⇒ **~5.3 h** (was ~22 h) - **~540/hr is an upper bound, not the expected rate**: it is the finest plausible quantum (one span per digest entry). The adapter chooses its cadence; a coarser one lands between 130 and 540. **Recommendation: keep `IO_LOG_MAX_ROWS = 1000` and replace the sentence's BASIS with the figures above.** A re-derivation may honestly land on the same number with a new basis, and this one does: the byte ceiling stays as derived (1250 rows × 16KB = 20MB pathological — the module refuses a second byte knob, and that refusal is right), and a ~1.85 h peak horizon still exceeds any live adapter poll loop by orders of magnitude. The failure mode of a too-short log is a gap, which is **the same failure an adapter must already tolerate** for a dead cursor: a new session seeds silently rather than replaying. **Fork for the gater** (I recommend the first): keep 1000 with the restated basis · or raise to 4000 to restore the ~7.5 h peak horizon, at a 68MB pathological per-endpoint ceiling. --- ## 5. Registry changes (authored before the code, per the traceability contract) - **NEW `REQ-IO-MIDTURN-SPAN`** — the `mid` attribute end to end: the CLI flag, the kind table of §1.5 including both named refusals, the frame attribute, the row field, the poll projection, and the additive N-1 reading of §1.3. `required_stages = ["doc", "impl", "unit", "int"]`. - **AMEND `REQ-IO-SEAL-SHORTFORM-GRAMMAR`** — one sentence: the bare case is reachable at the turn-close ingest, where end-of-output is text the parser holds; in a `mid` span an odd trailing marker is refused by name and mints nothing. Rulings 12-14 are otherwise verbatim. - **AMEND `REQ-IO-SHORTFORM-GATE` / the checklist wording** — the exactly-once-span contract replaces once-per-turn. - Registry hygiene: hand-edit, then **immediately** `traceable-reqs check` (exit 2 = the registry did not parse, and every later reading is vacuous), and **no `"` inside a title** — that is the exact character that killed the parse on 2026-08-25. ## 6. Test plan **Units** - `state_io_kind` — the whole table of §1.5 as one truth table, including the two refusal arms (the pure fn keeps its purity; `mid` is an argument). - `parse_seal_mints` under a mid-turn span: a pair mints; an odd trailing marker mints nothing and yields the named refusal; a pair PLUS an odd trailer mints the pair only (ruling 12's order preserved — pairing still runs first). - The same text at turn close mints the bare seal unchanged — **both polarities, one test file**, so the narrowing cannot be widened silently. - `render_json` emits `"mid": true` only when set, and `IoLogRow` round-trips it. - **The forward-compat unit §1.2 asks for**: a stored row carrying a field this binary does not know is read, not skipped — pinning the serde default as a DECISION, so a later `deny_unknown_fields` reddens a test instead of silencing a poll. **Integration** (real e2e legs per IR-67, never a `-p spt --bins` compile leg) - `api state busy --payload-stdin --mid` → the poll returns an `AGENT_OUTPUT` row with `mid`, the closing `idle` report returns one without it, and both bodies are verbatim and in order. - A mid-turn `@<…@>` dispatches at the mid-turn call (the latency property, measured not described). - `--mid` at `idle`, and `--mid` with no payload, both refuse by name. - `--no-fail-fast` on any run whose table claims siblings held. ## 7. Build order (after the gate) 1. Registry (§5) + `traceable-reqs check`. 2. `mid` through the five sites of §1.3, with §1.5's table. 3. The §3 bare-case arm and its refusal routing. 4. Units. 5. Doc sweep §4 + the rate basis. 6. e2e legs. 7. Both arms (Windows + kitsubito Linux), then the thin PR off current main, ff-only. Pool: re-claim from `.worktrees/ir57` with a `w2-238-midturn` label at build start (a claim is last-writer-wins, and every lane-identity arm speaks at the next BUILD, not at claim time). ## 8. What I need from the gate 1. **§3's decision (C)** — the centre of this document, and a declared narrowing of the gater lean. 2. **§4.1's fork** — keep 1000 with a restated basis (recommended), or raise to 4000. 3. **§1.3** — that mid-turn rows are additive and NOT version-gated, with the over-count named as the accepted N-1 degradation. 4. **§1.4's spelling** — `mid="1"` present-only, against `span="mid|final"`.