# RCA: Daemon Brain Rehashes Its Executable Every Heartbeat

**Issue:** https://github.com/SaberMage/spt-releases/issues/1  
**Reporter:** hertz  
**Date:** 2026-07-10  
**Affected:** spt 0.31.0; defect first shipped in v0.4.0  
**Status:** Root pinned; minimal process-lifetime digest cache recommended

## User-visible problem

On an otherwise idle four-vCPU Linux VPS, `spt daemon brain` continuously consumes approximately 40–55% of one CPU. `perf` attributes roughly 73–76% of samples to `sha2::sha256::compress256`. File tracing repeatedly observes:

1. `readlink("/proc/self/exe", ...)`;
2. opening and reading `/usr/local/bin/spt`;
3. creating `brain.ready.tmp.<pid-sequence>`;
4. renaming it over `brain.ready`.

The issue's five-second sample averaged 45.2% of one CPU. No daemon or endpoint lifecycle operation was performed during the original or source-level diagnosis.

## Pinned call chain

`crates/spt-daemon/src/brainproc.rs` contains the complete loop:

1. `BRAIN_HEARTBEAT = 500 ms` at line 63.
2. `run_brain` sleeps on that cadence at lines 249–250.
3. Every heartbeat unconditionally calls `write_ready(generation)` at line 287.
4. `write_ready` unconditionally calls `current_exe_hash()` at lines 355–360.
5. `current_exe_hash` performs `current_exe()`, reads the entire executable, and hashes all bytes at lines 365–374.
6. `write_ready` then atomically rewrites `brain.ready` at line 362.
7. `spt-store/src/atomic.rs:111+` writes a temporary file and renames it into place.

Nominal cadence is therefore:

`2 complete executable reads + 2 SHA-256 digests + 2 atomic brain.ready rewrites per second`.

The runtime report's approximately one observed sequence per second is a trace/sample observation, not the configured source cadence.

## Intent versus implementation

The D7-1 source comment at `brainproc.rs:365-370` states the intended cost explicitly:

> One exe read + hash per brain start.

The implementation put `current_exe_hash()` inside `write_ready`, but `write_ready` already served two roles:

- initial generation readiness publication after resume;
- recurring 500 ms readiness/liveness refresh.

That placement accidentally converted a once-per-process resident-image diagnostic into a heartbeat operation.

The digest is an additive `brain.ready.exe_hash` breadcrumb proving which binary bytes the brain generation actually runs. It protects seamless update/rollback validation and the D7 survival tests. The diagnostic itself is load-bearing; its heartbeat recomputation is not.

## History

- `0c954353b256f46e45a600be7a7fb1f3768e9d30`: D1 introduced recurring `brain.ready` heartbeat publication.
- `8859c9b1d004e5ab824a089eef4c07b4027df56b`: D6 generation-stamped readiness after session resume.
- `a52f8c6331e368777348a14c703363afc0585183`: D7-1a added `exe_hash` to `write_ready`.
- First released in v0.4.0; still present in v0.31.0.

The regression was introduced when D7's once-per-brain digest was composed into D1's recurring writer without separating immutable process identity from refreshable readiness.

## Falsified alternatives

### Update or stale-image reconciliation drives the load

Falsified. The digest call is unconditional on every idle heartbeat and does not inspect staged update state.

### Subnet meet-up or connection lifecycle activity triggers hashing

Falsified. `write_ready` runs after each heartbeat independently of network work. A net-less idle brain follows the same call path.

### Atomic JSON publication is the dominant CPU source

It causes avoidable disk churn but does not match the profile: approximately three quarters of CPU samples are in SHA-256, and source reads/hashes the entire binary before writing a small JSON record.

### Repeated hashing is required to detect executable replacement

Falsified by process semantics and update architecture. A running process's resident image is immutable for its lifetime. Seamless update terminates/replaces the old brain and starts a new brain process, which naturally computes a new digest. Rehashing the on-disk path from an old process can be actively misleading after rename/replacement because disk path identity and resident bytes can diverge.

This is also a breadcrumb-truthfulness defect, not merely performance: after an executable swap, a still-running old-byte process can re-resolve/re-read the replaced executable path and publish the new file's digest as though it described its resident image. The repeated read can therefore make the D7 diagnostic lie in the exact update incident class it was introduced to disambiguate.

## Correct minimal fix

Compute the resident executable digest once per brain process and reuse it for every `brain.ready` publication.

Suggested shape:

1. After brain process startup, compute `current_exe_hash()` once.
2. Hold `Option<String>` in process-local immutable state.
3. Pass the cached value to initial and heartbeat `write_ready` calls, or encapsulate generation/hash in a `ReadyPublisher`.
4. Keep best-effort behavior: if the one startup read fails, omit `exe_hash` for that generation rather than retrying twice per second.
5. A newly spawned brain generation recomputes its own digest exactly once.

Do not key a cache by filesystem metadata and do not invalidate it when the canonical executable path changes. The breadcrumb describes resident process bytes, not whichever bytes currently occupy the install path.

## Readiness-write scope

The minimal safe fix preserves the existing 500 ms atomic `brain.ready` refresh because comments and consumers treat it as a readiness/liveness breadcrumb. Removing or throttling that write is a separate design decision requiring a consumer audit and explicit liveness semantics.

After digest caching, the remaining two small atomic writes per second may still be worth reducing, but it is not necessary to close the reported SHA-256 CPU regression and should not be bundled without evidence.

## Deterministic regression seam

Refactor publication behind a process-lifetime object or injected digest source:

`ReadyPublisher { generation, exe_hash }`

Construction computes the digest; `publish()` only serializes the cached value and atomically writes.

Required tests:

1. **Hash count invariant** — construct one publisher, publish repeatedly, assert the injected digest function is called exactly once.
2. **Heartbeat reuse** — simulate initial publish plus multiple heartbeat publishes and assert every JSON body carries the same cached hash without new digest work.
3. **Best-effort failure** — a failed startup digest produces valid `{pid,generation}` readiness repeatedly with no retry storm.
4. **Generation replacement** — two publisher/process fixtures with different executable bytes compute once each and produce different hashes.
5. **Existing survival E2E** — retain the D7 fixture-A/fixture-B swap proof that a newly spawned brain reports the new resident hash.
6. **Atomic publication parity** — generation, pid, additive compatibility, temp-write/rename semantics, and supervisor promotion behavior remain unchanged.
7. **Complexity/performance contract** — hash operations are `O(brain process starts)`, not `O(heartbeats × executable size)`.

Avoid a flaky wall-clock CPU assertion as the primary CI gate. Use digest-call counters for deterministic behavior; keep a bounded Linux profiling/smoke acceptance to confirm idle CPU returns near sleep baseline.

## Operational expectation after fix

For a brain lifetime with `H` heartbeats and executable size `S`:

Before:

`hash work = O(H × S)`

After:

`hash work = O(S)` once per brain process.

At a 500 ms heartbeat, a 4.5-hour process currently performs approximately 32,400 complete binary hashes. The fix reduces that to one while preserving the resident-byte proof across update-driven process replacement.

## Agreed triage disposition

Doyle source-verified and accepted the RCA. The requirement title must carry the correctness invariant explicitly: **the brain readiness digest identifies immutable resident process bytes and must never be invalidated from replaced path metadata**.

Ratified implementation:

- OnceLock-style once-per-process digest capture in `current_exe_hash`;
- reuse the cached result on every heartbeat;
- a failed best-effort capture stays `None` for that process, with no retry storm;
- every new/rollback brain process computes its own fresh digest;
- preserve the 500 ms readiness-write cadence unchanged;
- correct the stale “one hash per brain start” implementation/comment composition in the same change.

Ratified gates:

1. injected digest counter equals one across initial plus `N` heartbeat publications;
2. failed capture stays absent without per-tick retries;
3. D7 fixture-A/fixture-B process-replacement E2E remains green and proves a new process publishes its own new hash;
4. readiness generation, atomic publication, and promotion semantics remain unchanged.

Scheduling: rider on **THE-FORKENING W4** (update-UX wave), built by Todlando after W3. The requirement will be minted registry-first at W4 dispatch. This report is canonical acceptance material.