# Phase 5: Deploy - Discussion Log

> **Audit trail only.** Do not use as input to planning, research, or execution agents.
> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.

**Date:** 2026-05-08
**Phase:** 05-deploy
**Areas discussed:** Env topology, CI/CD shape, Logs/observability, Secrets bootstrap

---

## Initial gray-area selection

| Option | Description | Selected |
|--------|-------------|----------|
| Env topology | Single prod vs separate staging+prod vs branch-deploys | ✓ |
| CI/CD shape (DEP-04) | Auto-deploy direct to prod / staging+promote / tag-based | ✓ |
| Logs destination (DEP-06) | Fly built-in vs external sink (Better Stack / Axiom / Logtail) | ✓ |
| Secrets + legacy seed bootstrap | BETTER_AUTH_SECRET, Tigris creds, Ed25519 keypair, localList.txt onto Fly | ✓ |

**User's choice:** All 4 areas.

---

## Env topology

### Q1 — Single Fly app or two-app staging+prod split?

| Option | Description | Selected |
|--------|-------------|----------|
| Two-app split | rebno-staging + rebno-prod, isolated Volume + Tigris each, ~2× fixed cost (still <$10/mo) | ✓ |
| Single prod-only | Cheaper, RESTORE.md validated against prod snapshot side-path | |
| Branch-deploys via fly machines | Single app, two machines, shared Volume namespace — SQLite WAL conflict risk | |

**User's choice:** Two-app split (Recommended).

### Q2a — Fly region for both apps?

| Option | Description | Selected |
|--------|-------------|----------|
| iad (Virginia, US-East) | STACK.md default for US-centric play | |
| ord (Chicago, US-Central) | Coast-balanced US, slightly worse EU than iad | ✓ |
| ams (Amsterdam, EU) | If returning playerbase trends EU | |
| You decide | Claude picks based on assumed geography | |

**User's choice:** ord (Chicago, US-Central).

### Q2b — Same region for staging and prod, or different?

| Option | Description | Selected |
|--------|-------------|----------|
| Same region | Latency profile + Tigris RTT translate 1:1 from RESTORE.md validation to prod | ✓ |
| Different region | Cheaper Tigris egress; validation results don't translate cleanly | |

**User's choice:** Same region (Recommended).

### Q3a — How does staging get refreshed/seeded with legacy accounts?

| Option | Description | Selected |
|--------|-------------|----------|
| Once at first deploy, persist | Mirrors prod ritual; staging dogfooded once before prod sees it | ✓ |
| Fresh-seed on every staging deploy | Tests migration ritual frequently; burns staging history | |
| Anonymized synthetic accounts only | No localList.txt on staging; loses migration smoke test | |

**User's choice:** Once at first deploy, persist (Recommended).

### Q3b — Should staging accept external traffic or be IP-restricted?

| Option | Description | Selected |
|--------|-------------|----------|
| Open WSS, hidden URL | Public endpoint, simplest, real localList.txt = minor exposure | |
| Allowlist via Fly proxy + token | IP-allowlist + STAGING_INVITE_TOKEN query param | ✓ |
| Tailscale-only | Strongest isolation; Phase 6 client testing needs Tailscale | |

**User's choice:** Allowlist via Fly proxy + token (Recommended).

---

## CI/CD shape (DEP-04)

### Q4 — Pipeline flow on push to main?

| Option | Description | Selected |
|--------|-------------|----------|
| main → auto-deploy staging → manual promote prod | Forces RESTORE.md exercise + Phase 4 carry-forward checks before prod | ✓ |
| main → auto-deploy direct to prod | Simplest; no staging dogfood between main and prod | |
| Tag-based prod, no auto-staging | Cheapest CI minutes; loses 'every merge dogfooded' property | |

**User's choice:** main → auto-deploy staging → manual promote prod (Recommended).

### Q5 — Where does trace:check hard-gate sit?

| Option | Description | Selected |
|--------|-------------|----------|
| Required PR check + verify-phase-N hook | Closes Phase 4 carry-forward DEP-04 hard-gate finding | ✓ |
| Standalone required PR check only | Single workflow; verify-phase-N skips it (no double-run) | |
| Soft warning only | Reverses Phase 4 carry-forward intent — not recommended | |

**User's choice:** Required PR check + verify-phase-4 hook (Recommended).

### Q6 — Promote-to-prod mechanism?

| Option | Description | Selected |
|--------|-------------|----------|
| Git tag push v*.*.* | Audit trail = git history; reuses staging-verified image SHA | ✓ |
| Manual gh workflow run deploy-prod.yml | UI button, less audit-trail discipline | |
| PR label deploy:prod | Mixes review and release control flows | |

**User's choice:** Git tag push v*.*.* (Recommended).

### Q7 — Rollback strategy?

| Option | Description | Selected |
|--------|-------------|----------|
| Re-tag prior version + redeploy | fly deploy --image <prior-sha>; Litestream side recoverability documented in RESTORE.md | ✓ |
| fly releases rollback | Fast but doesn't address Drizzle migration rollbacks | |
| Manual machine swap | Most control, most ops surface | |

**User's choice:** Re-tag prior version + redeploy (Recommended).

### Q8 — Drizzle DDL migrations on every deploy that touches schema?

| Option | Description | Selected |
|--------|-------------|----------|
| Pre-start in container entrypoint | drizzle-kit migrate before node dist; failure = crashloop = no traffic | ✓ |
| Separate ephemeral fly machine run before main deploy | More orchestration, finer error surface | |
| App-code boot phase (current Phase 4 pattern) | Same effect inside Node, not entrypoint | |

**User's choice:** Pre-start in container entrypoint (Recommended).

**Notes:** User initially answered with the legacy-account migration framing ("migration of legacy bno accounts and data only ever happens once. so there's no need to deeply integrate it"). Claude clarified two distinct migration concepts (Drizzle schema DDL vs one-shot legacy account import) and re-asked the schema-DDL question narrowly. User's clarified answer applies to schema DDL only; legacy-account migration stays as one-shot per env (Phase 4 D-17).

### Q9 — 30-min soak test (DEP-08) — manual or scripted in CI?

| Option | Description | Selected |
|--------|-------------|----------|
| Scripted via 2-client harness on staging | GitHub Action 30-min job vs rebno-staging post-deploy; reuses Phase 4 multi-client integ | ✓ |
| Manual run after every staging deploy | Realistic, slow, gates promote velocity | |
| Defer to Phase 6 first real client | Skip soak; CLI-08 demo doubles as soak | |

**User's choice:** Scripted via 2-client harness on staging (Recommended).

---

## Logs/observability (DEP-06)

### Q10 — Logs destination for pino JSON output?

| Option | Description | Selected |
|--------|-------------|----------|
| Fly built-in only | pino → stdout → Fly viewer; Fly default retention | |
| Fly built-in + Better Stack drain | External sink at v1, ~$10/mo, long retention + alerting | |
| Fly built-in + Axiom drain | Same shape as Better Stack, generous free tier, APL queries | |
| OpenTelemetry collector to self-hosted Loki later | Defer external sink to v2 | |
| Fly built-in + OTel + self-hosted observability stack in v1 | User-driven free-text answer | ✓ |

**User's choice:** Free-text — "fly built-in + otel + self-hosted observability stack in v1. not deferred."

### Q11 — Log levels + redaction strategy?

| Option | Description | Selected |
|--------|-------------|----------|
| info default, debug behind LOG_LEVEL env | Prod info, staging debug; pino redact per Phase 4 D-23 | ✓ |
| Verbose everywhere | Loud, costs more, best for first-month shake-out | |
| info on prod, debug on staging, trace gated | Three-tier with TRACE=1 flag for tick-loop microsecond timings | |

**User's choice:** info default, debug behind LOG_LEVEL env (Recommended).

### Q12 — Where does the OTel collector + observability stack live?

| Option | Description | Selected |
|--------|-------------|----------|
| Separate Fly app rebno-obs | Third Fly app holds OTel collector + observability backend; isolates blast radius | ✓ |
| Sidecar in each app machine | Lower hop latency; doubles compute, splits dashboard | |
| Sidecar collector + central self-hosted backend | Most flexible, more moving parts | |

**User's choice:** Separate Fly app rebno-obs (Recommended).

### Q13a — Observability backend stack? (initial framing)

| Option | Description | Selected |
|--------|-------------|----------|
| Grafana Alloy + Loki + Mimir + Tempo | Industry-standard 5-binary stack, ecosystem insurance | |
| Loki + Grafana + Prometheus only (logs+metrics) | Skip traces v1; simpler | |
| SigNoz | OTel-native single-binary-ish, ClickHouse storage | |
| OpenObserve | OTel-native, S3-backed (Tigris fit), single binary | |

**User's choice:** Free-text — asked for pricing + pros/cons between Grafana stack (option 1) and OpenObserve (option 4). Claude provided a comparison: both $0 software, Grafana stack ~$5-7/mo Fly compute (5 binaries, 2-4 GB RAM), OpenObserve ~$2-3/mo (single binary, 1 GB RAM); Grafana = ecosystem insurance, OpenObserve = minimal ops + better fit at <50 CCU.

### Q13b — Pick observability backend? (re-ask after comparison)

| Option | Description | Selected |
|--------|-------------|----------|
| OpenObserve | Single binary, OTel-native, Tigris-backed, ~$2-3/mo | ✓ |
| Grafana Alloy + Loki + Mimir + Tempo | Industry-standard, ~$5-7/mo, future-proof if scale grows | |
| SigNoz | Different tradeoff; more polished UI than OpenObserve, heavier than Grafana stack | |

**User's choice:** OpenObserve (Recommended for this scale).

### Q14 — What signals does the OTel collector emit?

| Option | Description | Selected |
|--------|-------------|----------|
| Logs (pino JSON via OTLP) | Replaces Fly built-in only for searchable + retained logs | ✓ |
| Metrics (process + Colyseus + game) | Powers basic ops dashboard from PROJECT.md Stage 5 | ✓ |
| Traces (Colyseus message handling) | Span per c2s message, span per HTTP auth; higher overhead | ✓ |
| OpenObserve native log ingestion only | Skip OTLP, stream pino direct; cheaper, less standard | |

**User's choice:** Logs + Metrics + Traces (all three).

### Q15 — rebno-obs access — same allowlist as staging, or separate?

| Option | Description | Selected |
|--------|-------------|----------|
| Same IP allowlist + obs auth password | Mirrors staging access; OpenObserve admin password via fly secrets | ✓ |
| Tailscale-only | Strongest isolation; tailnet required for mobile dashboards | |
| Public UI + strong auth (OAuth via GitHub) | OIDC; convenient for sharing; wider attack surface | |

**User's choice:** Same IP allowlist + obs auth password (Recommended).

---

## Secrets bootstrap

### Q16 — How does legacy/servers/enlyzeam-current/localList.txt get onto Fly?

| Option | Description | Selected |
|--------|-------------|----------|
| fly ssh sftp upload to /data, run, then delete | Plaintext never enters image, never enters git; manual ritual = good for one-shot | ✓ |
| Bake into image at build, immediately delete on first boot | Plaintext lands in image layers; layers persist in registry — bad | |
| Tigris-staged blob with one-shot fetch | More moving parts, signed URL coordination | |
| Rebuild localList from scratch on machine | Heredoc paste; tedious | |

**User's choice:** fly ssh sftp upload to /data, run, then delete (Recommended).

### Q17 — Where does BETTER_AUTH_SECRET (and other app secrets) come from?

| Option | Description | Selected |
|--------|-------------|----------|
| fly secrets set, generated locally with openssl rand | Different secret per env, encrypted at rest by Fly | ✓ |
| Generated at first boot, persisted to /data/.secrets | Survives redeploys; secret bound to volume | |
| Externally managed (1Password/Vault) | Heavier, overkill for this scale | |

**User's choice:** fly secrets set, generated locally with openssl rand (Recommended).

### Q18 — Ed25519 room-signing keypair generation + extraction?

| Option | Description | Selected |
|--------|-------------|----------|
| Auto-generate at first boot, extract pubkey via fly ssh | Private key never leaves machine; pubkey extracted once for Phase 6 Vite env | ✓ |
| Generate locally, fly secrets push private + commit pubkey | Easier pubkey distribution; private key passes through operator machine — weaker | |
| Per-env keypair (different staging vs prod) | Same as option 1 but explicit per-env split | |

**User's choice:** Auto-generate at first boot, extract pubkey via fly ssh (Recommended).

**Notes:** D-19 in CONTEXT.md folds the per-env-different-keys allowance into the recommended option (Phase 6 Vite already builds env-specific bundles).

### Q19 — Tigris credentials for Litestream replication?

| Option | Description | Selected |
|--------|-------------|----------|
| Per-app fly storage create | Auto-injected S3 env vars; per-bucket isolation | ✓ |
| Single shared bucket, prefixed by env | Cheaper but mistakes catastrophic | |
| Manual Tigris account, fly secrets injection | More control, more ops | |

**User's choice:** Per-app fly storage create (Recommended).

### Q20 — Secret rotation policy?

| Option | Description | Selected |
|--------|-------------|----------|
| BETTER_AUTH_SECRET rotation forces logout; Tigris keys rotate via Fly storage CLI; Ed25519 never rotates in v1 | On-demand only; documented procedures | ✓ |
| Quarterly forced rotation across the board | Cron-like every 90 days; heavy for v1 | |
| Defer policy entirely | Don't document rotation in v1 | |

**User's choice:** On-demand rotation, documented procedures (Recommended).

---

## Claude's Discretion

User explicit on each multi-choice; no full delegations. Remaining Claude-discretion items recorded in CONTEXT.md `<decisions>` §"Claude's Discretion":
- Dockerfile base image choice (`node:22-alpine` vs `node:22-bookworm-slim`) — STACK.md hints Alpine; planner verifies argon2 + better-sqlite3 prebuilt-binary availability before locking.
- OTel collector form factor (sidecar process vs in-process `@opentelemetry/sdk-node`) — planner picks lower-overhead at planning time.
- `flyctl` GitHub Action version pin — current mainline at plan time.
- OpenObserve version pin — current stable at plan time, drift-guarded by `lint-deploy-stack.mjs`.

## Deferred Ideas

No new deferrals beyond what's recorded in CONTEXT.md `<deferred>`. Multi-region, LiteFS, Postgres, autoscaling, OAuth, OAuth-on-OpenObserve, quarterly rotation, real Grafana dashboards, distributed tracing, automated pubkey distribution, Phase 6/7 client + asset work, trace:check `apps/**` scanner expansion all defer per the existing v1/v2/Phase-7 boundaries.