# PROVENANCE — gate arm r2, golden run 34481993681 Preserved by deployah, 2026-09-10. Measurements only; **causal classification is doyle's** and is not recorded here as settled. ## What this run is | field | value | |---|---| | run id | 34481993681, attempt 1 | | workflow | golden | | event | **push** (twohost unconditional on this path) | | ref | `refs/heads/golden/crossnode-294-r2` | | head sha | `a9e786b24fe4e541dcd394ef428f225d72f3024a` | | created | 2026-09-10T13:20:10Z | | purpose | THE GATE for v0.69.0 (releases#294) — a clean-candidate FULL-twohost green on a9e786b2 | The ref was created by pushing the OBJECT (`a9e786b2…:refs/heads/golden/crossnode-294-r2`), reported `* [new branch]`; ls-remote sha matched and tree `74fa640f78b0a588cd01b90c5898f9ea73e5dc08` == `a9e786b2^{tree}`, all verified BEFORE the run id was recorded. `golden/crossnode-294` already pointed at a9e786b2, so re-pushing that ref would have fired nothing. This arm restores the record run's **workflow CONFIGURATION** (push path, twohost unconditional). It does **not** restore prior machine state. ## Job table (RUN TERMINAL; re-observed 14:15:39Z) | job | id | status | started | completed | |---|---|---|---|---| | traceability | 102886817360 | success | 13:20:13Z | 13:20:24Z | | changes | 102886817577 | success | 13:20:26Z | 13:20:32Z | | n1-gate (Windows, hfenduleam) | 102886952292 | success | 13:20:35Z | 13:27:56Z | | n1-gate (Linux, kitsubito) | 102886952334 | success | 13:49:46Z | 13:51:30Z | | test (Linux, kitsubito) | 102886952398 | **success** | 13:20:35Z | 13:49:44Z | | test (Windows, hfenduleam) | 102886952516 | **FAILURE** | 13:27:58Z | 13:58:10Z | | twohost-b | 102900252121 | success | 13:58:13Z | 14:13:21Z | | twohost-a | 102900252212 | success | 13:58:13Z | 14:13:57Z | | notify | 102906061685 | success | 14:14:00Z | 14:14:17Z | twohost-a/b are `needs: test` (golden.yml:1035, :1213) and carry `!cancelled()`, so the Windows red neither ended the run nor freed the boxes — both ran to completion after it. **TERMINAL POPULATION: NINE jobs**, now stated from the completed job list rather than a live one (an in-flight list understates by construction; this one is no longer in flight). **Both twohost legs PASSED** — twohost-b success, twohost-a success — as did `notify`. The ONLY red in the run is test(Windows). Run-level `conclusion` reads `failure`. **Recorded, never scored from**: the run-level `status` field lied at every read on the record run. The verdict above is assembled from per-job conclusions. ## Windows leg — the RED `test-windows-102886952516.log`, 578,896 bytes, pull rc=0, stderr empty. Applied CONTRACT FINAL (doyle, 2026-09-10, three passes; supersedes the earlier "two Summaries carrying the same population = VOID" clause, which is **withdrawn**): count Summaries, state the EXPECTED execution count, map each Summary to its phase BY THE LOG'S STRUCTURAL BOUNDARY, population corroborates only, surplus beyond the mapping = VOID. - **Summary count: 1. Expected: 2** (Phase A + Phase B). The mapping does **not** close, and the direction is a **DEFICIT, not surplus** — a missing execution, not the poison shape. - The single Summary (line 4208) belongs to Phase A by structural position: the `test-phase-a` step group opens at line 760, `Starting 3381 tests across 138 binaries (1 test and 90 binaries skipped)` at line 798. - Population corroborates: `Summary [ 756.587s] 3381 tests run: 3380 passed (6 slow, 5 leaky), 1 failed, 1 skipped` — 3380 + 1 = 3381, full population accounted. - **Why there is no Phase B, read structurally and not inferred from duration:** immediately after the Summary the log carries `error: test run failed`, then `##[error]Process completed with exit code 1`, and the next step group is FLOOR_END rather than a second nextest. `--no-fail-fast` let Phase A finish its whole population; the step's nonzero exit then ended the job before Phase B started. - Second, independent corroboration of the same deficit: this job carries **four** CI-CENSUS phases (start, pre-reap, post-settle, end); both bare arms carry **ten**, because the prerelink cycles belong to Phase B. ### Failure classes, SUFFIX ALLOWED FAIL + LEAK [ 2 occurrences LEAK [ 5 panicked at 1 **Reconciliation closes:** the two `FAIL + LEAK` occurrences are ONE unique cell (line 2785 in-stream, line 4209 in the final recap) = the Summary's own `1 failed`. > **The banked hazard fired live, recorded as a measurement and not a near-miss:** > `grep -c 'FAIL \['` on this log returns **0**. A bare FAIL-prefix grep reports NO failures on a > job that failed. The reconciliation is the check; the grep is not. ### The cell FAIL + LEAK [ 55.685s] (1987/3381) spt-daemon::sync two_tier_sync_lands_and_gate_refuses_server_side crates\spt-daemon\tests\sync.rs:109:5 panic: sync stream never appeared at the responder's broker test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2 filtered out; finished in 55.40s Cell stderr also carried: NTP_TOTP_UNCORRECTED: all NTP servers unreachable — ceremony clock = raw system clock CONN_LIFECYCLE: conn=3 role=brain stream-subscriber stream=1 … event=stream-sub-attach stream=1 from_seq=0 [REQ-CONN-POISON-ATTRIBUTION] **Named, NOT classified.** This is the same cell both bare arms tracked as a SLOW PASS — arm 1 `PASS [ 91.375s] (1993/3346)`, arm 2 `PASS [ 94.000s] (2019/3381)`. Whether that makes it Finding 1 is doyle's call; no reproduction is asserted here. Disk floor PASS: `free_bytes=62937976832` against `floor_bytes=34359738368`. ## Linux leg — CLEAN `test-linux-102886952398.log`, 682,195 bytes, pull rc=0, stderr empty. - **Summary count: 2. Expected: 2.** Mapping closes, no surplus. - line 4264 — `Summary [ 44.868s] 3360 tests run: 3360 passed (2 leaky), 1 skipped` (Phase A) - line 4508 — `Summary [1183.807s] 223 tests run: 223 passed (1 slow), 0 skipped` (Phase B) - Distinct populations and durations, each mapping to its phase by structural position. - **Zero** failure-class rows under the suffix-allowed enumeration. - Populations match arm 2's Linux exactly (3360 / 223). - **The cell that failed on Windows passed here in the same run:** `PASS [ 1.523s] (1751/3360) spt-daemon::sync two_tier_sync_lands_and_gate_refuses_server_side` The red is Windows-only within this run. ## Claims WITHDRAWN during this harvest (recorded, not silently dropped) Kept because a provenance file that shows only the surviving claims teaches nothing about how the surviving ones were arrived at. All three were caught by todlando. 1. **"It did not run to completion"** — asserted from duration alone (30m12s against arm 2's 66m19s). Duration is not execution extent. Replaced by the structural reading above: Phase A completed its full population; Phase B never started. Different statement, and this one is measured. 2. **"There is no slowdown"** — derived from a PHASE TOTAL (756.587s vs 791.935s). A phase total is nextest's ELAPSED WALL TIME for the phase — **not** a sum of the parallel cell durations (terminology corrected by todlando; the original wording here said "a sum over 3,381 cells" and was wrong). The limitation is unchanged by the correction: an elapsed wall time cannot locate a delay inside any one cell. What is measured is that the two totals are within 35s. Internal timing is UNMEASURED. 3. **"55.685s is faster than either pass — the opposite of a starvation shape"** — a failing run and a passing run traverse DIFFERENT PATHS (this one exits at the line-109 assert the moment the wait expires; a pass continues past it), so the durations are not comparable quantities and their ordering carries no starvation signal in either direction. Same class as the banked duration-over-a-timeout rule: compare against the constant, not against the fast case. The wait's own budget in `sync.rs` has NOT been read out; until it is, 55.685s means nothing. ## Control run, reported although it REFUTES the lead that motivated it The end-of-job reap census lists six FOREIGN `spt.exe` from `…\AppData\Local\spt-core\bin\spt.exe` (the agent fleet's own daemons) live on hfenduleam during the job. This looked like exactly the non-GitHub load that the earlier job-level contention measurement on the record red explicitly could NOT rule out. r2 CI-CENSUS phase=start box_procs=643 family_total=16 unscoped=16 arm 1 CI-CENSUS phase=start box_procs=640 family_total=15 unscoped=15 arm 2 CI-CENSUS phase=start box_procs=635 family_total=14 unscoped=14 **NARROW form of the finding (the broad form is withdrawn):** the foreign process POPULATION was comparable across all three arms, so an ELEVATED FOREIGN PROCESS COUNT is not what separates them. A count is not a load — identical counts are consistent with very different CPU, disk and port pressure, and six idle daemons census the same as six busy ones. **Contention is NOT excluded.** ## Scope limits - Every job-state figure above is stamped at its observation time. An unstamped running count in a durable record is stale on arrival. - The run was **not terminal** when this file was written; twohost-a/b were still in_progress. Nothing here is a RUN verdict — these are two PER-JOB verdicts. - A RED is not automatically a Finding 1 reproduction. The cell and its path are named above so that comparison can be made by someone else, against the record red's own cell. ## Instrument defect found during this harvest Poll `bl5wrywo2` (armed 13:21:03Z) stayed ALIVE and tracked state correctly — its ticks show 6 completed at 13:58:29Z — but it **only notifies on EXIT**, and its predicate waits for ALL jobs terminal. twohost-a/b materialized 3s after the Windows leg died and carry `!cancelled()`, so the predicate could not be satisfied for another hour. The red sat unreported for ~12 minutes while status was being given as "nothing terminal", from readings up to 30 minutes stale. Its `concl=[]` field was **empty at every tick including after a job concluded `failure`**, so even on exit it would have handed back an empty conclusion list, which reads as clean. Repair requirements (todlando's, adopted): notify on INDIVIDUAL JOB TRANSITIONS rather than whole-run completion; and validate the repaired conclusion collector against THIS already-failed Windows job before trusting it. An empty conclusion collection is invalid evidence, not clean status.