---
phase: "05-deploy"
plan: "08"
subsystem: "server-observability"
tags: [otel, sdk-node, pino-otel-transport, fail-soft, traces, metrics, logs, dep-06]
dependency_graph:
  requires: ["05-01", "05-02"]
  provides: ["otel-init.ts OTLP SDK boot", "log.ts multi-target transport (gated)", "otel-init unit+int tests"]
  affects: ["apps/obs (Plan 09 — OTLP receiver target)", "verify-phase-5.mjs (Plan 14 — test gate)"]
tech_stack:
  added:
    - "@opentelemetry/sdk-node ^0.217.0"
    - "@opentelemetry/auto-instrumentations-node ^0.75.0"
    - "@opentelemetry/exporter-trace-otlp-http ^0.217.0"
    - "@opentelemetry/exporter-metrics-otlp-http ^0.217.0"
    - "@opentelemetry/exporter-logs-otlp-http ^0.217.0"
    - "@opentelemetry/instrumentation-pino ^0.63.0"
    - "@opentelemetry/sdk-logs ^0.217.0"
    - "@opentelemetry/sdk-metrics ^2.7.1"
    - "@opentelemetry/sdk-trace-base ^2.7.1"
    - "pino-opentelemetry-transport ^1.1.0"
  patterns:
    - "OTel SDK boot via node --import (side-effect ESM module)"
    - "D-12 fail-soft: try/catch around entire SDK init"
    - "Gated pino multi-target: enabled only in production with OTEL_EXPORTER_OTLP_ENDPOINT set"
    - "Independent SIGTERM hook for sdk.shutdown() (separate from sigterm.ts)"
key_files:
  created:
    - apps/server/src/otel-init.ts
    - apps/server/test/otel-init.test.ts
  modified:
    - apps/server/src/log.ts
    - apps/server/package.json
    - pnpm-lock.yaml
decisions:
  - "pino-opentelemetry-transport pinned to ^1.1.0 (not ^3.0.0): v3.0.0 requires pino@^10.0.0; server uses pino@^9; v1.1.0 supports pino@^8.21||^9"
  - "Added @opentelemetry/sdk-{trace-base,metrics,logs} as direct deps (not just transitive): otel-init.ts directly imports BatchSpanProcessor, PeriodicExportingMetricReader, BatchLogRecordProcessor from these packages; TypeScript requires explicit resolution"
  - "pino transport gate: OTEL_EXPORTER_OTLP_ENDPOINT && NODE_ENV === 'production' — prevents worker-thread requirement in test environments (Phase 4 vitest suite stays stable)"
metrics:
  duration: "~30 minutes"
  completed: "2026-05-08T11:35:00Z"
  tasks_completed: 3
  files_created: 2
  files_modified: 3
---

# Phase 05 Plan 08: OTel SDK Boot + pino Multi-Target Transport Summary

**One-liner:** NodeSDK boot via `node --import` with OTLP-HTTP exporters for traces/metrics/logs, fail-soft try/catch per D-12, and gated pino-opentelemetry-transport that only activates in production.

## Objective

Wire the in-process OpenTelemetry SDK into `apps/server` (otel-init.ts) and add a parallel OTLP log transport to pino (log.ts). Both wires fail-soft. Closes DEP-06 [impl] + [unit] + [int] stages.

## Tasks Completed

| Task | Name | Commit | Files |
|------|------|--------|-------|
| 1 | Add OTel + pino-otel deps | a7cfdc5 | apps/server/package.json, pnpm-lock.yaml |
| 2 | otel-init.ts + log.ts | 27fbfd5 | apps/server/src/otel-init.ts (new), apps/server/src/log.ts |
| 3 | otel-init.test.ts | 7e6738d | apps/server/test/otel-init.test.ts (new) |

## Final OTel Package Versions Pinned

Verified via `npm view <pkg> version` at execution time (2026-05-08):

| Package | Plan Pin | Verified Current | Final Pin |
|---------|----------|-----------------|-----------|
| @opentelemetry/sdk-node | ^0.217.0 | 0.217.0 | ^0.217.0 |
| @opentelemetry/auto-instrumentations-node | ^0.75.0 | 0.75.0 | ^0.75.0 |
| @opentelemetry/exporter-trace-otlp-http | ^0.217.0 | 0.217.0 | ^0.217.0 |
| @opentelemetry/exporter-metrics-otlp-http | ^0.217.0 | 0.217.0 | ^0.217.0 |
| @opentelemetry/exporter-logs-otlp-http | ^0.217.0 | 0.217.0 | ^0.217.0 |
| @opentelemetry/instrumentation-pino | ^0.63.0 | 0.63.0 | ^0.63.0 |
| @opentelemetry/sdk-logs | (transitive) | 0.217.0 | ^0.217.0 |
| @opentelemetry/sdk-metrics | (transitive) | 2.7.1 | ^2.7.1 |
| @opentelemetry/sdk-trace-base | (transitive) | 2.7.1 | ^2.7.1 |
| pino-opentelemetry-transport | ^3.0.0 (plan) | 3.0.0 current | **^1.1.0** (pino@^9 compat — see Deviations) |

## log.ts Gating Logic

Transport is enabled only when both conditions are true:
- `process.env.OTEL_EXPORTER_OTLP_ENDPOINT` is set and non-empty
- `process.env.NODE_ENV === 'production'`

**Rationale:** pino's multi-target transport spawns worker threads. In the vitest test environment (`pool: 'forks'`, `maxWorkers: 1`) this works but is unnecessary overhead and risks flakiness. The gate keeps Phase 4 tests using plain pino-to-stdout and only enables the OTel transport in production Fly.io deployments where `NODE_ENV=production` and the OTLP endpoint are both set (Plan 02 fly.toml sets both).

The redact config object (all 34 paths + `censor: '[Redacted]'`) is defined once and spread into the same `pino({...})` call as the transport, satisfying Pitfall 10: all transport targets (stdout + OTLP) share the identical redact pipeline.

## Notes for Downstream Plans

### Plan 09 (apps/obs — OpenObserve)
- Must accept OTLP-HTTP at port 5080, path prefix `/api/default`
- Full paths: `/api/default/v1/traces`, `/api/default/v1/metrics`, `/api/default/v1/logs`
- The server constructs these URLs explicitly as `${OTEL_EXPORTER_OTLP_ENDPOINT}/v1/{traces,metrics,logs}` where the endpoint is `http://rebno-obs.flycast:5080/api/default`

### Plan 11 (soak verification)
- After 30 minutes of soak traffic, query OpenObserve REST API to confirm at least 1 trace record landed
- Suggested query: `GET /api/default/traces?limit=1` with OpenObserve auth headers
- Also verify no `session_token` or `password` plaintext appears in any log record (Pitfall 10 / T-DEP-OTEL-LEAK threat)

### Plan 14 (verify-phase-5.mjs)
- Must include `pnpm --filter @rebno/server test` in its steps array
- This runs the full unit test suite including `otel-init.test.ts` as the DEP-06 gate
- Command for targeted run: `pnpm --filter @rebno/server exec vitest run test/otel-init.test.ts`

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 2 - Missing Critical Functionality] Added @opentelemetry/sdk-{trace-base,metrics,logs} as direct deps**
- **Found during:** Task 2 typecheck
- **Issue:** `otel-init.ts` imports `BatchSpanProcessor` from `@opentelemetry/sdk-trace-base`, `PeriodicExportingMetricReader` from `@opentelemetry/sdk-metrics`, and `BatchLogRecordProcessor` from `@opentelemetry/sdk-logs`. These were only transitive deps; TypeScript `NodeNext` module resolution requires them to be declared directly.
- **Fix:** Added 3 packages to `apps/server/package.json` dependencies and ran `pnpm install`
- **Files modified:** apps/server/package.json, pnpm-lock.yaml
- **Commit:** 27fbfd5

**2. [Rule 1 - Bug] Downgraded pino-opentelemetry-transport from ^3.0.0 to ^1.1.0**
- **Found during:** Task 1 pnpm install
- **Issue:** pino-opentelemetry-transport@3.0.0 has peer dependency `pino@^10.0.0`; the server uses `pino@^9`. Plan cited ^3.0.0 without checking the pino version compatibility window.
- **Fix:** Used `^1.1.0` which declares peer `pino@^8.21.0 || ^9.0.0`. v1.1.0 is the latest pre-pino-v10 release. The OTLP transport functionality is identical.
- **Files modified:** apps/server/package.json, pnpm-lock.yaml
- **Commit:** a7cfdc5

## Pre-existing Test Failures (Out of Scope)

Two pre-existing test failures were observed during the `otel-init.test.ts` run (they appeared because the test runner also picked up other test files in the same run):

- `test/persistence.test.ts` — `better-sqlite3` native binding not compiled in the worktree environment (unrelated to this plan)
- `test/admin-stubs.test.ts` — pre-existing assertion about "clipboard" string matching (unrelated to this plan)

The `otel-init.test.ts` tests themselves: **4/4 passed**. `log.test.ts` (Phase 4 redact): **2/2 passed**.

## Threat Surface Scan

No new public network endpoints, auth paths, or file access patterns introduced. The OTLP exporter targets `rebno-obs.flycast:5080` (Fly.io internal 6PN network — not public). Redact pipeline verified to cover all pino targets simultaneously per Pitfall 10 and STRIDE entry T-DEP-OTEL-LEAK.

## Known Stubs

None — otel-init.ts is fully wired; log.ts transport is gated but functional in production.

## Self-Check: PASSED

- [x] `apps/server/src/otel-init.ts` exists and contains `[impl->REQ-DEP-06]`
- [x] `apps/server/src/log.ts` contains `[impl->REQ-DEP-06]`
- [x] `apps/server/test/otel-init.test.ts` exists and contains `[unit->REQ-DEP-06] [int->REQ-DEP-06]`
- [x] Commits a7cfdc5, 27fbfd5, 7e6738d all present in git log
- [x] 4/4 otel-init tests pass
- [x] 2/2 log.test.ts tests pass (no regression)
- [x] otel-init.ts line count: >60 lines (actual: ~85 lines)
- [x] All 7 required strings present in otel-init.ts: NodeSDK, getNodeAutoInstrumentations, OTLPTraceExporter, OTLPMetricExporter, OTLPLogExporter, OTEL_EXPORTER_OTLP_ENDPOINT, try, catch, process.on('SIGTERM'), sdk.start(), sdk.shutdown
- [x] All required strings in log.ts: [impl->REQ-DEP-06], redact, password, session_token, transport, pino-opentelemetry-transport, OTEL_EXPORTER_OTLP_ENDPOINT