Skip to content
GRPNR.

Case studies & experience reports

Sources: research/02-web-research.md (verified sweep, 2026-07-18), research/06–10 (industry blueprint batch), research/12 (fleet-monitor market scan). Dates noted; agentic-era reports ≤18 months old.

Large-scale agentic migration programs

Classic + agentic production era

Who Scale Verdict What broke / what they’d change Source (dated)
Bun (2026) 1M LOC JS→native, 64 concurrent agents, <2 weeks, $165k Success — the published ceiling for concurrency 19 escaped regressions; verification was the bottleneck, not generation 02-web-research (2026)
Airbnb test migration (2025) 3.5k Enzyme→RTL files, 6 weeks Success “Retry with feedback beats prompt perfection”; context curation (which files) was the quality lever; escalate context 40k→100k tokens per attempt 02-web-research (2025)
Anthropic internal (2025–26) Parity-harness pattern; build-daemon serialization Pattern source Characterization tests recorded BEFORE transform; “any change = bug”; serialize builds through one daemon per app 02-web-research
1Password B5 (2025) Multi-million-line Go monolith; MustBegin migration = 3,000+ call sites in hours Success via manifest→template→playbook→worktrees Engineering time shifted to analysis tools + playbooks, not coding; ULID-assumption incident forced full-session rollback research/10
Groupon’s own estate (found 2026-07-18) VIS3 strangler + identity-service migration Both died mid-flight, rotting in code Strangler debt without a retirement ledger = permanent wrappers; the #1 in-house cautionary tale analysis/legacy/SYNTHESIS §2

PRAISE / PAIN summary — agent fleets on migrations

  • PRAISE: bounded concurrency (≤64) with strong verification works and is cheap ($165k/1M LOC); deterministic manifests + playbooks turn weeks into hours; retry-with-feedback loops are robust.
  • PAIN: verification is always the bottleneck; agent tangling base rate 53.9%; unguided agents raise smell counts (Claude Code 179→215/commit unguided); AI PRs wait 4.6× longer for review, 32.7% acceptance; nothing published validates >64 concurrent, let alone 1000.

Fleet monitoring (candidate 3)

  • Market absence (negative result, confirmed twice, 2026): no product covers 50–100 sessions with task-level metrics (rework rate, cost/merged-unit, queue depth). Terragon shut down 2026-02; Conductor macOS-only; sniffly stale 11mo; Vibe Kanban 27.4k★ but upstream dead (Apache 2.0 snapshot — design donor only).
  • Adoptable base (2026): Claude Code native OTel (CLAUDE_CODE_ENABLE_TELEMETRY=1) → Collector → Prometheus+Loki → Grafana; fork ColeMurray/claude-code-otel (469★, stale-but-correct, pin); Grafana dashboard 25255 as base; Claude-Code-Usage-Monitor (8.5k★ active) + ccusage for quota; claude-squad (8.1k★) for ≤20 tmux supervision.
  • Anthropic Agent View (research preview 2026, v2.1.139+): one sortable list, needs input floats top, inline reply — “you become the integration point, not the bottleneck.” The operator UX model to copy.
  • Cost attribution gotcha: OTEL_RESOURCE_ATTRIBUTES="worktree=…,agent=…" (the often-cited CLAUDE_CODE_RESOURCE_ATTRIBUTES does not exist — fact-check caught it). OTEL_LOG_USER_PROMPTS stays OFF (credential leak).

Human-in-the-loop / question triage (candidate 2)

  • HumanLayer — the only purpose-built product — deprecated (2026). Its two primitives (require_approval, human_as_tool) are the build-thin spec.
  • langchain-ai/agent-inbox — only live off-the-shelf reference; LangGraph-coupled; validates the inbox shape, nets out build-thin on own infra.
  • Agent Teams (Anthropic experimental, 2026): inbox-per-agent JSON mailboxes, task self-claim with file locks; its anti-forgery rule (relayed “human approved” = untrusted) goes into the triage design.
  • Negative result: semantic dedup, per-stakeholder rate limits, and overload management for agent-question volume are unsolved in public literature — whoever builds it is first.
  • In-house precedent: old-estate partner-onboarding skill’s hard gate (“any gap stops the run, posts exact missing fields to Jira”) — the pattern already institutionalized at Groupon.

Verification / dual-run harnesses (candidate 4)

  • Impact-analysis-selected tests: ~70% regression reduction — the single best-evidenced mitigation in the migration literature (02-web-research).
  • Characterization/parity-before-port: Anthropic pattern; corroborated by GitHub-scale migration reports. No off-the-shelf generic “does A behave like B” product exists (negative result).
  • Zaruba’s own reconciliation design (_documentation/backend/ledger-design.md): three independent oracles (provider settlement vs order-store vs contract-check), replay-capped, “dry-run must have an owner” — an in-house, already-accepted instance of the pattern to generalize.
  • Legacy-side caveat: EOL runtimes (Rails 3.2/JRuby 1.7) mean shadow-recording needs a legacy-runtime lab (pinned Docker images) — engineering cost unique to this estate (analysis/legacy).

Knowledge graphs for agent context (candidate 5)

  • Industry blueprint claims 120× token reduction (3.4k vs 412k tokens/session) for graph-backed blast-radius queries (research/07, vendor-grade evidence — treat as directional).
  • Our own fleet run empirically hand-built the graph for 26 repos (162 nodes/327 edges) in one pass — the extraction is cheap; the question is productization value, not feasibility.
  • Zaruba’s greenfield has zero code-graph tooling and his doctrine consumes telemetry, not code structure [E1] — demand signal for a code KG is weak on the greenfield side, real on the legacy-audit side.

Declarative rewrite engines (candidate 6)

  • GritQL/ast-grep excellent for in-place codemods at scale (research/07; Bun/1Password used deterministic manifests similarly).
  • But: Zaruba’s doctrine ports no legacy code [E1]; MBUS-era Ruby is rebuilt, not transformed. The old-estate 1→N Encore split (where codemods DO apply) is a different program in a different repo, currently out of scope [E8].

MBUS bridge (candidate 9)

  • Legacy reality: 16/33 components on MBUS (analysis/legacy).
  • Greenfield reality: MBUS actively rejected — Pub/Sub + provider webhooks instead [E4]. A bridge would serve only shadow-comparison taps during dual-run, i.e., a harness feature, not an app.