# Phase 3: Diagnostic Features - Context

**Gathered:** 2026-04-12
**Status:** Ready for planning

<domain>
## Phase Boundary

Add diagnostic analysis features to captured frame sequences: delta highlighting to show what changed between frames, idle frame compression to eliminate redundant data, and animated GIF export for an alternative visual format. Covers: DIAG-01, DIAG-02, DIAG-03.

</domain>

<decisions>
## Implementation Decisions

### Delta highlighting (DIAG-01)
- **D-01:** Compare consecutive frame pixel buffers and create a semi-transparent red overlay on regions that differ above a configurable threshold
- **D-02:** Pixel comparison uses per-pixel RGB distance calculation; regions exceeding threshold get highlighted
- **D-03:** Delta overlay is composited onto each frame BEFORE grid compilation — each grid cell shows "frame + highlighted changes from previous frame"
- **D-04:** First frame has no delta overlay (no previous frame to compare against)
- **D-05:** Threshold is fixed at a sensible default (e.g., 10% pixel difference) — not user-configurable in v1 to keep the API simple
- **D-06:** `delta_highlight: boolean` parameter added to `start_capture` tool (default false)

### Idle frame compression (DIAG-02)
- **D-07:** Compare consecutive frames for near-identity (same pixel comparison as delta, but checking if ALL pixels are below threshold)
- **D-08:** Consecutive identical frames are collapsed into a single grid cell with label "unchanged for N.Ns" showing total idle duration
- **D-09:** The collapsed cell uses the first frame of the identical sequence as its image
- **D-10:** Compression happens BEFORE grid compilation — the frame array is reduced, then the smaller array is laid out in the grid
- **D-11:** `compress_idle: boolean` parameter added to `start_capture` tool (default false)
- **D-12:** Both delta_highlight and compress_idle can be enabled simultaneously — compression runs first, then delta highlighting on the compressed sequence

### Animated GIF export (DIAG-03)
- **D-13:** Use @skyra/gifenc for GIF encoding (per tech stack decision in CLAUDE.md)
- **D-14:** GIF export triggered by `gif_export: boolean` parameter on `start_capture` (default false)
- **D-15:** When gif_export is true, the completed session produces both grid JPEG AND animated GIF
- **D-16:** GIF exposed as a second MCP resource at `capture://{sessionId}/gif` with mime type `image/gif`
- **D-17:** GIF frame delay calculated from actual elapsedMs between frames (not uniform timing)
- **D-18:** GIF frames resized to reasonable dimensions (max 800px wide) to keep file size manageable for LLM consumption
- **D-19:** GIF generation happens after grid compilation (both from the same frame array, possibly compressed)

### Tool parameter design
- **D-20:** All three features are optional boolean flags on start_capture: `delta_highlight`, `compress_idle`, `gif_export`
- **D-21:** Flags stored in CaptureConfig and passed through to the processing pipeline
- **D-22:** get_capture_status returns `gifUri` field when gif_export was enabled and capture is complete
- **D-23:** Grid resource and GIF resource are independent — agent can fetch either or both

### Claude's Discretion
- Exact pixel comparison algorithm (simple RGB distance vs perceptual difference)
- Delta highlight color (red suggested, but any high-contrast overlay works)
- GIF color quantization strategy (256-color palette handling)
- Whether to add gif_quality parameter or hardcode sensible defaults
- Internal buffer handling for delta computation (in-memory vs streaming)

</decisions>

<canonical_refs>
## Canonical References

**Downstream agents MUST read these before planning or implementing.**

### Existing Implementation
- `src/processing/grid-compiler.ts` — Grid compilation pipeline (must integrate delta/compression before compilation)
- `src/processing/timestamp-overlay.ts` — Timestamp overlay pattern (reference for overlay compositing)
- `src/types.ts` — CaptureConfig, CaptureFrame, StartCaptureInputSchema (need extending with new flags)
- `src/server.ts` — Tool registration, resource registration (need GIF resource)
- `src/capture/scheduler.ts` — Frame collection (frames pass through to processing pipeline)

### Prior Phase Context
- `.planning/phases/01-core-capture-pipeline/01-CONTEXT.md` — Grid sizing decisions (1568px max, JPEG quality)
- `.planning/phases/02-window-and-region-targeting/02-CONTEXT.md` — CaptureConfig extension pattern

### Technology
- @skyra/gifenc — GIF encoding library (per CLAUDE.md tech stack)
- sharp — Image processing (pixel comparison, overlay compositing, resize)

</canonical_refs>

<code_context>
## Existing Code Insights

### Reusable Assets
- `compileGrid()` in grid-compiler.ts — Takes CaptureFrame[], produces JPEG buffer. New processing steps plug in before this
- `addTimestampOverlay()` in timestamp-overlay.ts — Pattern for overlaying text/graphics onto frame buffers with sharp
- CaptureFrame interface — Has `buffer: Buffer` (PNG) and `elapsedMs: number`, perfect for frame comparison
- Resource template pattern in server.ts — Pattern for adding `capture://{sessionId}/gif` resource

### Established Patterns
- Tool registration with zod schemas in server.ts
- CaptureConfig extended with new fields (established in Phase 2)
- Processing pipeline: capture -> store frames -> process -> compile grid -> expose as resource

### Integration Points
- `src/types.ts`: Add delta_highlight, compress_idle, gif_export to CaptureConfig and StartCaptureInputSchema
- `src/processing/`: New files for delta-highlighter.ts, idle-compressor.ts, gif-exporter.ts
- `src/server.ts`: Wire new config fields, add GIF resource template
- `src/processing/grid-compiler.ts`: Call delta/compression before grid layout (or pass pre-processed frames)

</code_context>

<specifics>
## Specific Ideas

- Delta highlighting should be obvious enough for an LLM to notice in a grid image — subtle differences won't work
- Idle compression directly reduces grid size, making the important frames more visible
- GIF export gives agents a different modality — animated view vs static grid comparison
- Processing pipeline order: raw frames -> compress idle -> delta highlight -> compile grid / export GIF

</specifics>

<deferred>
## Deferred Ideas

None — discussion stayed within phase scope

</deferred>

---

*Phase: 03-diagnostic-features*
*Context gathered: 2026-04-12*