02 — Discovery: Fleet Monitor
Brainstorm, problem, stakeholders, journeys, stories. Evidence tags: VERIFIED (source+date), PRIOR-RESEARCH (dated doc, not rechecked this pass), ASSUMPTION (reasoned, unverified). Numbers carry MEASURED/MODELED/DESIGN passports. Where this doc disagrees with 01-initial-recommendation.md, it says so inline — no silent alignment.
0. A fact that changes the frame
Two different fleet execution patterns exist and the packets show only ONE has actually run in production:
- Pattern A — N independent Claude Code sessions (separate OS processes, e.g. one per wezterm pane), each claiming ledger units. This is what
01-initial-recommendation.mdand the work-ledger doc both assume (“10→64 concurrent Claude sessions… claim units”), whatfleetopsdiscovers (~/.claude/sessions/<pid>.json+/procliveness, VERIFIED R4 2026-07-18), and what Anthropic’s Agent View backgrounds (VERIFIED R2 2026-07-18). - Pattern B — one Claude Code session (Fable) fanning out Task-tool subagents. This is what actually ran today: the 34-agent legacy-profiling wave (25 repo profilers + 8 sub-profilers + 1 merge, 0 errors, 15 min, ~2.1M subagent tokens — VERIFIED R4, sourced to
docs/llm-runs.md/analysis/legacy/SYNTHESIS.md, 2026-07-18). Run-log notation is “Claude Code session subagents (Fable orchestrating),” not N separate top-level sessions.
ASSUMPTION, not verified this pass: Task-tool subagents run inside the parent process, not as independently-PID’d processes — if true, fleetops’s /proc-based discovery and Agent View’s session backgrounding see none of them; both tools would be structurally blind to the one fleet pattern actually used today. This is the single highest-leverage thing to check before writing code (see §6, feasibility risk #1). Every downstream reuse verdict in this doc (fleetops, Agent View) is conditioned on which pattern the next real wave uses.
1. Brainstorm — does a monitor need to exist, and where
| Rung | What it is | Holds at | Verdict |
|---|---|---|---|
| (a) Do nothing — tmux + eyeballs | No tooling, operator tab-cycles panes | N ≲ 8–10 (worktree-management lore, explicitly labeled “community lore, unverified tier” in research/02-web-research.md §7 — PRIOR-RESEARCH, do not upgrade its confidence) |
Holds only below that ceiling. The 34-agent wave already blew past it — eyeballing 34 anything is not a real option regardless of pattern A/B. |
| (b) Pure first-party Agent View | Anthropic’s claude agents needs-input list + inline reply |
Single machine, single operator, Pattern A only | Does not hold alone, and may not apply at all if the fleet runs Pattern B (§0). Also gated to Pro/Max/Team/Enterprise/API — not Free (VERIFIED R2). No cross-machine (Tomas), no cost join. Correction to 01: 01-initial-recommendation.md’s “Simplest useful version (a)” names Agent View as the thing to adopt for the needs-input list. fleetops (VERIFIED R4, already built, WSL2/wezterm-native, already reads the same session files, already jumps to the pane) does the same job locally without the --bg/daemon model or the plan tier gate — prefer reusing fleetops over adopting Agent View, pending the Pattern A/B check above. |
| (c) Pure Grafana adopt | Aggregate dashboards over OTel metrics | Never, as the primary surface | Confirmed by both the original doc and this pass: operators work a queue, not a wall. Grafana belongs underneath, not instead of, a needs-input list — no new evidence overturns this. |
| (d) Kanban UI | Board with columns per state | Nowhere | Reject, explicitly. Two independent reasons converge: (1) 01-initial-recommendation.md already removed it — the ledger owns state, a board duplicates it; (2) VERIFIED today (R2, 2026-07-18) Vibe Kanban — the only donor named in research/12 — has gone quiet since Bloop’s shutdown (announced 2026-04-10), and no fork has taken over stewardship (top forks checked: 0–1 stars, pushed_at identical to upstream, i.e. unmodified mirrors). research/12’s “steal Vibe Kanban’s UI shape” suggestion is superseded by both the later doc’s decision and today’s OSS-health check. This closes contradiction (e)#1 from the R7 packet in favor of 01, explicitly. |
| (e) Thin custom layer (the 001/01 verdict) | OTel→Prometheus→Grafana + hook-ingest + ledger join, four metrics | Holds, but the shape is thinner than 01 assumed | See below — this is the surviving rung, revised. |
| (f) Full custom control room | NOC wall, historical analytics, alert zoo | Nowhere | No evidence anywhere in five source packets supports this for a 2-operator shop. Not re-litigated. |
Revision to rung (e), the actual finding of this pass: 01-initial-recommendation.md scoped “build” as hook-event ingest → SQLite → new local web page, plus a fresh docker-compose OTel/Prometheus/Grafana stack. R4 (VERIFIED 2026-07-18) found most of the data-collection half of that already exists, split across five personal TUIs at ~/tui/: fleetops (session/status board — this job, done), ghmonitor (GitHub Actions + deployments, Go, alpha — this is directly Tomas’s CI ask, near-zero new build), bridge (prod log tail), ground-control (local dev-server status), tokenomics (Claude subscription usage %, not per-unit cost attribution — does not substitute for the ledger’s cost/merged-unit metric). Separately, R5 (VERIFIED 2026-07-18) found Tomas’s workflow-execution and human-task-queue data is already exposed as authenticated REST list/filter endpoints in his own workflow-management service — a monitor needs one minted API token and two polls, no new infra, no watcher code in his repo.
Net effect on the 001 verdict: ADOPT-plus-thin-layer still holds as the shape, but “thin layer” now means an aggregation/join view over tools that already exist (fleetops + ghmonitor + ledger-when-built + Zaruba’s two REST endpoints), not a new ingest pipeline. The genuine gap is single-pane aggregation and the ledger-derived metrics (rework rate, cost/merged-unit) that no existing tool computes — not data collection. Whether the OTel→Prometheus→Grafana stack is still needed for v1, or whether tokenomics+ledger cost columns cover the cost signal without standing up a new collector, is an open build question, not a discovery-doc call — flagged for the design doc.
2. Problem discovery
Core problem (as stated in 01, re-tested here): at fleet scale, the operator cannot tell which agents are blocked/awaiting-input without tab-cycling; blocked agents idle silently; there is no cost/quality accounting per merged unit.
| Claim | Classification | Basis |
|---|---|---|
| Tab-cycling breaks down as fleet size grows | STRONGLY-SUPPORTED | Not measured via interview/incident log this pass, but self-evident from scale: a 34-agent wave ran today (VERIFIED R4) and no tool exists that would let one operator watch 34 terminals simultaneously without a status board. |
| Blocked/needs-input agents currently go undetected until the operator happens to check | HYPOTHESIS | No incident record found in any packet of an agent silently stalling and costing wall-clock. The one documented wave (34 agents, 15 min, 0 errors) is a clean-run counter-example, not supporting evidence. This claim is inherited from 01-initial-recommendation.md’s pre-research judgment, not independently confirmed. Treat as the thing the 1-week trial must actually observe, not a settled fact. |
| No cost/quality accounting exists per merged unit today | CONFIRMED | Absence-of-capability, verified by inventory: tokenomics gives subscription-level usage %, not per-unit cost; the work-ledger (the thing that would carry per-unit cost) is decided-not-built (R7d, PRIOR-RESEARCH); no other tool in the R4 scan computes this. |
| That absence is actually costing Robert money/time today | UNSUPPORTED-ASSUMPTION | No budget overrun, no “we spent $X on a unit that got reverted” incident is documented anywhere in the source packets. The pain is architecturally plausible (matches Bun-case-study-scale industry pattern, PRIOR-RESEARCH) but not evidenced locally. |
| Trust collapse from one false needs-input signal sends the operator back to tmux permanently | HYPOTHESIS | This is 01’s own stated “biggest usability risk,” argued from design principle (Norman), not from an observed incident — no prior false-positive event is on record. Real risk to design around, not a measured fact. |
| Tomas’s deployment/workflow visibility gap is real and currently zero | CONFIRMED | Tomas explicitly asked 2026-07-18 16:52 GChat (PRIOR-RESEARCH, cited in 01 line 5); R5 confirms today there is no existing watcher anywhere against his execution/task/CI surfaces — this is a true, not assumed, gap. |
| GitHub Actions / CI status is the right proxy for “watch Tomas’s deployments” | UNKNOWN | R5 flags that both repos’ Actions workflows only cover frontend Cloud Run deploys and an inert infra skeleton; the backend Encore.ts apps deploy via Encore Cloud’s own pipeline, invisible to GitHub Actions entirely, and Encore Cloud’s API/webhook surface was not checked this pass (R5 item 4, explicit gap). If backend deploys are what Tomas actually means by “watch my deployments,” current 3a/3b watch-surface plans (GH Actions) would miss the thing he cares about most. |
Silent-stall problem: downgraded from CONFIRMED to HYPOTHESIS by this pass — no observed incident, only architectural plausibility at scale. Keep in scope (it’s what stall detection exists to catch) but do not report it to Tomas as a solved-and-measured problem; report it as the trial’s primary open question.
Cost-accountability problem: the capability gap is CONFIRMED; the pain from the gap is UNSUPPORTED-ASSUMPTION. Both facts belong in the MVP justification — build for the gap, don’t claim measured pain that doesn’t exist yet.
3. Stakeholders
| Stakeholder | Goals | Frustrations | Success definition | Influence |
|---|---|---|---|---|
| Robert — operator | See every blocked/needs-input unit in one glance; unblock in <60s; know fleet cost as it accrues | Tab-cycling; false status (worse than none); tools that add ceremony he’ll skip under load | “0 blocked” rendered large, believed without checking | Full — sole builder + primary user, decides scope |
| Tomas — consumer + watched party | Read-only visibility into Robert’s fleet health (daily digest); wants his own execution/task/deploy surface watched too | Being asked to instrument his own code for someone else’s monitor; unclear what “watched” even covers (frontend? backend? Temporal?) | Daily digest arrives, is accurate, costs him zero new code | Medium — his ask defines a second scope (§2), but he is not a dashboard user and has no write access |
| Fleet agents (Claude Code sessions/subagents) | Machine users: need a place to raise a hand (block on question) and have it surface fast | N/A — not a UX stakeholder in the human sense, but a wrong signal about them (false stall/false needs-input) directly damages trust in the tool | Their blocked state is visible within the polling interval, no false positives | Indirect — their telemetry is the raw material, but they have no say in design |
| Groupon confidentiality/security context | Nothing internal (prompts, code, workflow contents) leaks outside the localhost boundary | OTEL_LOG_USER_PROMPTS accidentally on; a monitor phoning home; Zaruba API tokens over-scoped |
Local-only by construction, VIEWER-role tokens only, no telemetry ever leaves the box | Constraint-setter, not a person — enforced by design rule, not negotiation |
| Future third operator | Onboard without archaeology; same visibility Robert has, day one | Tribal knowledge encoded only in Robert’s head/tmux muscle memory | Can look at the same board and understand fleet state within minutes of joining | Low today (hypothetical), but a real reason to keep the design legible rather than personally-tuned |
4. Journeys
Operator happy path (glance → triage → unblock) Goal: return to decision work with every blocked unit answered. Steps: glance at board → see N needs-input, rest recede → open top item inline → answer or route → confirm cleared → repeat until “0 blocked.” Friction: any step that requires a terminal tab switch. Failure state: operator can’t find the source pane for a flagged item (metric says “needs input,” nothing points at where). Metric (MODELED, target from 01): operator-minutes per interrupt <1.
Telemetry-not-arriving (the onboarding-failure equivalent) Goal: fleet unit’s status reaches the board at all. Steps: agent starts → hook/OTel emits → collector ingests → board reflects within polling interval. Friction: silent drop anywhere in that chain looks identical to “agent is fine and idle” — there is no distinct “I don’t know” state today (Norman’s “honest confidence signal, not a binary lie,” 01 line 33, PRIOR-RESEARCH). Failure state: a genuinely-stalled agent and a telemetry-dead agent render the same (“no news”), and the operator cannot tell which. This is the sharpest design risk in the whole product — worth its own acceptance test before v1 ships (see §5).
False-stall / false-needs-input (trust incident) Goal: never happens, but if it does, recovery must be fast and legible. Steps: board flags unit X → operator checks → unit was actually fine → operator now doubts the next flag too. Friction: no confidence/provenance shown on the flag (is this MEASURED from a hook event or MODELED from a heuristic timeout?). Failure state: operator reverts to tmux “just to be sure,” permanently, on the theory articulated in 01’s biggest-risk line — this journey exists to be tested, not assumed, since §2 found no prior incident evidence. Metric: false-positive rate observed during the 1-week trial (currently zero data).
Tomas daily digest Goal: Tomas gets an accurate, low-effort read on Robert’s fleet + his own watched surfaces without opening a dashboard. Steps: digest compiled (fleet health + Zaruba execution/task summary) → delivered via Google Chat (house standard, PRIOR-RESEARCH R7f) → Tomas reads in <2 min. Friction: digest includes a surface that’s actually unwatched (e.g., backend Encore Cloud deploys — CONFIRMED unwatched, §2) and implies coverage that doesn’t exist. Failure state: digest silently omits a category Tomas assumes is covered — this is the same “false confidence” failure as the trust-incident journey, aimed at Tomas instead of Robert. Metric: digest sent daily, zero omission complaints.
Wave post-mortem (cost/quality review) Goal: after a wave (e.g., the 34-agent profiling run), answer “what did this cost, and was it worth it” without archaeology. Steps: wave completes → cost/merged-unit and rework rate pulled from ledger+monitor → compared against the wave’s actual output. Friction: this journey is entirely blocked on the work-ledger, which is decided-not-built (CONFIRMED absence, §2) — cost/merged-unit and rework rate both require the ledger join per the canonical four-metric definition (PRIOR-RESEARCH R7c). Failure state: post-mortem falls back to “felt fast, no numbers.” Metric: cost/merged-unit answerable yes/no (01’s own concept-test metric, line 28) — today, no.
5. User stories
Format: As <persona> I want <action> so that <outcome>. Tag: MVP / post-MVP / rejected.
Critical path
| # | Story | Acceptance | Tag |
|---|---|---|---|
| 1 | As Robert I want one list of units needing my input, sorted to the top, so I never tab-cycle to find one | List renders within the polling interval; needs-input unit disappears within one cycle of being answered | MVP |
| 2 | As Robert I want a distinct “telemetry not arriving” state (not silently rendered as “fine”) so a dead pipe never looks like an idle agent | A unit with no signal for >2× expected interval renders a third state, not green/red | MVP |
| 3 | As Robert I want to answer a blocked unit inline without switching to its terminal pane | Reply path exists that doesn’t require finding/activating the source pane | MVP |
| 4 | As Robert I want every status flag to show whether it’s MEASURED (hook event) or MODELED (heuristic/timeout) so I can calibrate trust per-flag | Passport visible on hover/inline per 01’s binding UI rule | MVP |
Activation
| # | Story | Acceptance | Tag |
|---|---|---|---|
| 5 | As Robert I want to point the monitor at a wave and see it populate with zero manual wiring | A new wave’s units appear without per-wave config | MVP |
| 6 | As Robert I want to confirm which fleet pattern (A: sessions, B: subagents) a wave is running before I trust the board | A one-line check (“N sessions discovered” vs “wave declared M units, 0 sessions found”) surfaces the mismatch instead of silently showing zero | MVP — directly answers §0 |
| 7 | As Tomas I want to receive the daily digest without installing or configuring anything | Digest arrives in Google Chat; zero setup on his side | MVP |
Operations
| # | Story | Acceptance | Tag |
|---|---|---|---|
| 8 | As Robert I want cost/merged-unit visible per wave | Requires ledger join; blocked until ledger ships — degrade to “unavailable,” not fabricated | post-MVP (ledger-gated) |
| 9 | As Robert I want rework rate visible per wave | Same ledger dependency as #8 | post-MVP (ledger-gated) |
| 10 | As Robert I want queue depth (units waiting, per state) visible without the ledger | Can run off session/OTel data alone per 01’s open question 6 — no ledger dependency | MVP |
| 11 | As Robert I want Tomas’s workflow_executions FAILED/TERMINATED/stuck-IN_PROGRESS counts on the same board |
Poll against surface 1a (R5), VIEWER-role token, no new infra | MVP |
| 12 | As Robert I want Tomas’s tasks queue joined to stuck executions so “WAITING_FOR_APPROVAL” shows who owns it |
Poll surface 2 (R5), join on workflowId/runId |
post-MVP (adds real value but not required for v1 trial) |
| 13 | As Robert I want GitHub Actions run status for Tomas’s frontend once the PAT lands | Blocked purely on PAT provisioning (R5); trivial once unblocked | post-MVP (blocked) |
Rejected
| # | Story | Why rejected |
|---|---|---|
| 14 | As Robert I want a kanban board of unit states | Duplicates the ledger’s own state ownership; donor project (Vibe Kanban) has no living fork (§1) |
| 15 | As Robert I want auto-remediation (auto-kill/restart stuck agents) | Explicitly out of MVP per 01; auto-acting on an unverified stall signal compounds the false-positive risk instead of reducing it |
6. Riskiest assumptions (Cagan/Torres) → 1-week trial coverage
| Assumption | Type | Risk if wrong | Tested by the 1-week trial? |
|---|---|---|---|
| The fleet actually runs Pattern A (sessions) often enough that fleetops/Agent-View-style tooling applies | Feasibility | fleetops (and the whole board concept as scoped) is blind to the dominant real pattern; rebuild around Task-tool subagent introspection instead | Yes, directly — story #6 is designed to surface this on day one of the trial |
| Silent stalls are a real, frequent operator pain (not just architecturally plausible) | Desirability | Building stall detection solves a problem that doesn’t actually occur at current fleet sizes | Yes — trial measures hours of silent stall discovered late (01’s own concept-test metric, line 28) |
| A needs-input board is trusted enough to replace tmux-checking, not just supplement it | Usability | Operator keeps both habits, board becomes overhead, not routing | Yes — trial measures operator-minutes per interrupt before/after |
| Cost/merged-unit and rework rate require the ledger and cannot be approximated from session/OTel data alone | Feasibility | If false, v1 could ship these metrics without waiting on the ledger; if true (current assumption), stories 8–9 stay blocked | Partially — trial can confirm queue-depth/stall run ledger-free (story #10); it cannot resolve the ledger-gated metrics since the ledger won’t exist during the trial window |
Watching workflow_executions + tasks (R5 surfaces 1a/2) is what Tomas means by “watch my deployments/workflows” |
Desirability | Digest reports the wrong thing as covered while the surface he actually cares about (backend Encore Cloud deploys, CONFIRMED unwatched) stays dark | No — the trial as scoped watches Robert’s fleet; validating this against Tomas requires showing him the digest and asking, not inferable from telemetry alone. Flag as a pre-trial conversation, not a trial output |
| Two-operator digest delivery via Google Chat is sufficient — no dashboard needed for Tomas | Viability | Building any Tomas-facing UI beyond a digest is wasted effort | Partially — trial can observe whether Tomas asks for more once he sees a week of digests |
Bottom line for this pass: the 001/01 ADOPT-plus-thin-layer verdict survives, but two of its load-bearing assumptions move — the reuse target for the needs-input list shifts from Agent View toward fleetops (pending the Pattern A/B check), and “thin layer” now means aggregating existing personal tools plus two REST polls against Zaruba’s app, not standing up a new ingest pipeline. The riskiest open item is not technical — it is whether the fleet pattern the whole design assumes (independent sessions) is the one actually run in practice.