UX/UI specification — dual-run parity harness

2026-07-18 · Template §9. Scope note (stated once, applies throughout): this is an internal CLI/CI/report product whose primary user is a machine (a fleet agent claiming a work-ledger unit). Every generic-SaaS UX section (onboarding funnels, growth loops, visual branding, mobile layouts) is either reinterpreted internally or explicitly cut — cuts are marked [SCALED DOWN] inline rather than silently omitted. “Screens” in this product are artifact types: JSON verdicts, markdown evidence bundles, terminal output, one Grafana panel, one CI annotation. There is no web app.

1. UX vision

Intended feeling: “the verdict is trustworthy and checkable.” Not “fast,” not “friendly,” not “beautiful” — those are all subordinate to checkable. A parity harness that agents or humans stop trusting is worse than no harness: the program reverts to vibes at 99.999% stakes (initial-recommendation.md §10, “diff noise → false DIVERGED → trust collapse → bypass”). Every design choice below is subordinate to this one feeling.

Time-to-first-value target: first real verdict (PARITY or DIVERGED, not a scaffolding demo) on a pilot service, from a cold clone of the harness repo. DESIGN target: same day (record → verify → evidence bundle on one long-tail pilot endpoint chain, per the fastest-prototype plan in initial-recommendation §11 — “days, not weeks” for the harness’s own bring-up; a single pilot repo’s first verdict should land inside one of those days once the harness itself exists). This is a DESIGN target, not yet MEASURED — no pilot has run.

Trust principles (three, load-bearing, not decoration):

Self-replay falsification is the harness’s own audit trail. Record → replay against the same legacy instance must yield PARITY, always, as a precondition the harness checks on itself before any real verdict is trusted (initial-recommendation §11). No published tool in this space discloses a false-positive rate (D6 Q2 — Diffy’s own precision was never published anywhere found) — so the harness cannot borrow anyone else’s track record. It manufactures its own evidence by testing itself the same way it tests the rewrite.
The normalization ledger is confession, not configuration. Every excluded field (timestamps, UUIDs, ordering, float precision) is listed with a reason, in every bundle, every time — never a hidden .harnessignore. A parity report that hides its exclusions is a lie with good typography (initial-recommendation §7, Rams lens).
The replay recipe travels with the verdict. Every evidence bundle carries the exact command to reproduce it locally. A verdict nobody can independently re-run is an assertion, not evidence — and this program’s currency is receipts (source + window + confidence, per the context brief’s binding terminology section).

2. Information architecture

2.1 CLI surface

Five verbs, deliberately minimal — flag creep is a named risk (§6). No verb is added without a unit economics case at the next flagship gate.

harness record   <adapter> <unit>     # capture legacy traffic (HTTP + shadow-tap) into a fixture set
harness verify    <adapter> <unit>     # replay fixtures against candidate, emit verdict + evidence bundle
harness explain   <verdict-id>          # render a stored verdict's evidence bundle to terminal/markdown
harness lab up|down <stack>            # boot/stop a pinned legacy-runtime lab image
harness selfcheck <adapter>            # record→replay against legacy itself; must yield PARITY or the harness itself fails

Global flags kept to the load-bearing few: --adapter, --unit, --out <dir> (bundle destination), --ci (machine-only stdout, JSON, no ANSI/progress chrome), --threshold <pct> (override the adapter’s default parity floor for a single run — always logged as a diverging-from-default event, per Norman error-prevention below). No --verbose tiers, no --format fanout — one bundle shape, always. [SCALED DOWN]: no shell completion, no interactive wizard, no config UI — a workspace-package harness.config.ts per adapter is the only configuration surface, version-controlled like everything else the program treats as an auditable scope decision.

2.2 JSON verdict schema (sketch)

{
  "verdictId": "orders-adapter/checkout-flow/2026-07-18T09:14:02Z",
  "adapter": "orders",
  "unit": "checkout-flow",
  "verdict": "DIVERGED",              // PARITY | DIVERGED | UNVERIFIABLE
  "parityRate": 0.99987,              // fraction of compared traffic classes at PARITY
  "threshold": 0.995,                 // functional-tier floor, adapter-declared (program config, ADR-5) —
                                       // NOT the money gate's 99.999%, which is Zaruba's live reconciliation
                                       // threshold (prd.md §13/Option B), never a per-unit verify number
  "sampleSize": 4200,                  // fixtures compared this run
  "counterexample": {                  // present only on DIVERGED; smallest known-failing case
    "fixtureId": "checkout-flow/fx-0231",
    "field": "response.body.totalCents",
    "legacy": 4899,
    "candidate": 4901
  },
  "unverifiableReason": null,          // populated only on UNVERIFIABLE, see 3.4 taxonomy
  "normalizationRulesApplied": ["ts.createdAt", "id.requestId", "order.eventOrdering"],
  "evidenceBundle": "s3://.../bundles/orders-checkout-flow-2026-07-18.tar.zst",
  "replayRecipe": "harness verify orders checkout-flow --fixtures fx-set-2026-07-18 --candidate-rev a1b2c3d",
  "selfCheckPassed": true,             // this run's own self-replay falsification result
  "generatedAt": "2026-07-18T09:14:02Z"
}

Every field above is either directly consumed by the work-ledger row (§2.4) or exists so a human/agent never has to open the bundle to know whether to open the bundle. counterexample is deliberately the single smallest failing case, not the full diff set — the full set lives behind evidenceBundle (progressive disclosure, §4).

2.3 Exit-code contract

Deterministic, script-safe, no ambiguity for CI or fleet-agent branching:

Code	Meaning
`0`	PARITY
`1`	DIVERGED
`2`	UNVERIFIABLE (see reason taxonomy, §3.4)
`3`	Harness internal failure (lab didn’t boot, recorder crashed) — never conflated with a verdict
`4`	Self-check failed — the harness does not trust its own last run; treat as `3`-severity, not a verdict

Exit code 3/4 must never be silently retried into a 0 — a flaky lab boot is an UNVERIFIABLE or a harness bug, never grounds for a cached stale PARITY.

2.4 Evidence-bundle structure

Fixed order, top to bottom, because reading order is the disclosure design (Ive lens — one verdict per screen-height, everything else behind links):

bundle/
  verdict.json           # §2.2, always read first
  counterexample.md       # smallest failing case, human-readable, ~1 screen
  normalization-ledger.md # every excluded field + reason, always present even when empty ("none excluded")
  replay-recipe.sh        # the exact command(s) to reproduce this bundle locally
  diffs/                  # full diff set, one file per divergent fixture — opened only on demand
  fixtures/               # content-addressed request/response + shadow-tap captures, masked per PII policy
  lab-manifest.json       # pinned image digests + boot config used for this run

counterexample.md and normalization-ledger.md are the two files a human ever reads by default; diffs/ and fixtures/ are for the rare deep audit (QSA/SOX, or a builder who doesn’t trust the counterexample selection).

2.5 Work-ledger verdict-row fields

The Encore-side surface — rows only, per architect correction #4 (context brief §“Decided harness shape”). Minimal, matches what promotion-gating and the Grafana panel need and nothing more:

Field	Purpose
`unitId`	joins to the fleet’s existing work-ledger unit
`adapter`	which domain adapter produced this row
`verdict`	PARITY / DIVERGED / UNVERIFIABLE
`parityRate`, `threshold`	the two numbers that decide promotion
`evidenceBundleUrl`	link out, never inlined — work-ledger stores pointers, not report content
`selfCheckPassed`	promotion gate refuses `false` regardless of `verdict`
`verifiedAt`, `harnessVersion`	audit trail — which kernel version produced this claim

Structural rule, not a UI affordance: the PR-lane promotion query in work-ledger requires a row with verdict = PARITY AND selfCheckPassed = true to exist for a unit. There is no “promote anyway” button anywhere in this product — see §4, error prevention.

2.6 The one Grafana panel

Exactly one, fed from work-ledger verdict rows, three series, no drill-down widgets beyond native Grafana tooltip:

Parity coverage — % of in-scope units per adapter with a current (non-stale) PARITY row.
Divergence rate — DIVERGED rows / total verify runs, rolling 7-day, per adapter.
Verify latency — p50/p99 of harness verify wall-clock time, per adapter (catches legacy-lab boot-pain regressions before they become the “fixtures go stale” risk named in initial-recommendation §10).

[SCALED DOWN]: no exec dashboard, no per-builder leaderboard, no cost-tracking panel here (token/ engineer-time economics belong to the program’s existing cost surfaces, not a new one). One panel, because a second panel is the first symptom of dashboard sprawl this document explicitly refuses (§5).

3. Core “screens” = artifact types

3.1 Verdict JSON

Objective: the single machine-consumable fact a fleet agent branches on.
Primary action (by the consumer): read verdict, branch: PARITY → claim next step; DIVERGED → mint a triage question with counterexample attached; UNVERIFIABLE → branch per reason (3.4).
Hierarchy: verdict first field logically (schema order matches read order); counterexample only on DIVERGED; everything else is provenance.
Empty/error/loading equivalents: there is no “loading” state for a stored verdict — it’s written atomically or not written at all (no partial-verdict files, ever; a crashed verify run leaves no verdict.json, only exit code 3). “Empty” = no verdict exists yet for a unit → the work-ledger promotion query fails closed (§2.5), which is the empty-state UX: silence reads as “not yet safe,” never as an implicit pass.
Analytics events (internal, for the harness’s own health, not a customer product): verdict.emitted {adapter, unit, verdict, parityRate, durationMs}, selfcheck.failed {adapter} — feeds the Grafana panel (§2.6) and nothing else. [SCALED DOWN]: no funnel/retention analytics — there is no funnel.

3.2 Evidence bundle (markdown-rendered)

Objective: let a human decide, in under 30 seconds of reading, whether a DIVERGED verdict is a real regression, an acceptable difference, or a missing normalization rule.
Primary action: the builder’s decision (real regression / acceptable / add normalization rule) — and that decision is itself written back as an audited event (§4), never a silent close.
Hierarchy: verdict → smallest counterexample → normalization ledger → replay recipe → full diffs behind links (§2.4 order is the hierarchy).
Empty/error/loading: a PARITY bundle still exists and is still readable — “verdict: PARITY, compared N fixtures, 0 divergent, self-check passed” is not omitted just because there’s nothing to triage; auditors need PARITY evidence bundles just as much as DIVERGED ones (QSA/SOX DONE-row). An UNVERIFIABLE bundle replaces counterexample.md with a reason-taxonomy block (3.4) — never a blank file.
Analytics events: bundle.opened {verdictId, actor: human|agent}, bundle.decision {verdictId, decision: regression|acceptable|normalization-rule-proposed, actor}.

3.3 Divergence triage view (markdown)

Objective: when a DIVERGED verdict is minted into the triage service (context brief: “triage service dedups agent questions”), give the triage consumer exactly enough to dedup and route, no more.
Primary action: dedup against existing open questions, route to a builder or auto-close as a known/accepted normalization case.
Hierarchy: one-line summary (field, legacy vs candidate values) → link to full bundle. This view is intentionally a stub of the evidence bundle, not a duplicate render — one source of truth.
Empty/error/loading: if triage-dedup matches an existing open question, the triage view says so explicitly (“duplicate of TRIAGE-114, not re-opened”) rather than silently dropping the new divergence — silent drops are how noise erodes trust (§6 top risk).
Analytics events: triage.question.minted {verdictId}, triage.question.deduped {verdictId, matchedId}.

3.4 Lab status output (`harness lab up/down`)

Objective: tell the operator (human or CI) whether the legacy-runtime lab is in a state that can produce trustworthy fixtures, before a record run wastes time recording against a half-booted stack.
Primary action: proceed to record, or fix the reported blocker.
Hierarchy: single status line first (READY / BOOTING / FAILED: <reason>), then per-container health detail behind --verbose equivalent (piped to a log file by default, not stdout — CI logs are already noisy enough without full Docker boot chatter on the happy path).
UNVERIFIABLE reason taxonomy (used identically in verdict JSON, evidence bundles, and lab status — one taxonomy, not three):
- LAB_BOOT_FAILED — pinned image didn’t come up (manifest-v1 deprecation risk flagged in D3 is a live candidate cause for the oldest stacks)
- FIXTURE_STALE — recorded fixtures predate the current legacy or candidate revision by more than the adapter’s staleness window
- MBUS_TAP_UNAVAILABLE — shadow-tap couldn’t attach (no OSS tool covers this leg per D1 §11 — expect this reason to be exercised early and often)
- NORMALIZATION_RULE_MISSING — a field diverges in a pattern the ledger has no rule for yet (distinct from DIVERGED: this is “we don’t know if this counts,” not “this is wrong”)
- SELF_CHECK_FAILED — the harness’s own falsification precondition (§1) didn’t hold; no verdict is trustworthy until this clears
- SCOPE_DARK — the unit touches a capability class the audit doctrine hasn’t scoped yet (Resque batch/cron jobs are the confirmed DARK case per the context brief §D4 digest — structurally uncapturable by an HTTP-proxy recorder)
Analytics events: lab.boot.completed {stack, durationMs, result}, lab.boot.failed {stack, reason}.

3.5 CI annotation

Objective: the PR-surface glance — does this unit’s harness gate pass, in the one line a reviewer actually reads on a CI check list.
Primary action: click through to the evidence bundle only if not PARITY.
Hierarchy: ✓ orders/checkout-flow — PARITY (99.7%, self-check passed) or ✗ orders/checkout-flow — DIVERGED (1 counterexample) → <bundle link> or ⚠ orders/checkout-flow — UNVERIFIABLE (MBUS_TAP_UNAVAILABLE) → <lab status link>. One line, one glyph family (✓/✗/⚠ only — no color-only signaling, accessibility-basic, non-negotiable per the “never simplify away” floor). The percentage shown here is the functional-tier parityRate (§2.2) against its adapter-declared floor — this is the lightweight per-unit verdict, not the money gate; the ≥99.999% ledger-reconciliation number belongs only to Zaruba’s live reconciliation plane (prd.md §13/Option B) and never appears on a per-unit CI annotation.
Empty/error/loading: no annotation posted = no harness run occurred for this PR — CI config, not this product, decides whether that blocks merge; the harness never posts a placeholder “pending” comment that could be mistaken for a pass.
Analytics events: none beyond what’s already captured at verdict-emission (3.1) — the annotation is a rendering of an existing event, not a new one.

4. Interaction design

Determinism/idempotency. harness verify given the same fixture set + candidate revision must produce byte-identical verdict.json (modulo generatedAt/verdictId). This is not a nicety — it’s what makes the self-check (§1) meaningful at all: if replay isn’t deterministic, self-replay-against- legacy failing PARITY can’t be distinguished from the harness itself being flaky. harness record is explicitly not idempotent (each run may observe new legacy behavior) — the CLI never implies otherwise; re-running record always creates a new fixture-set ID, never silently overwrites one (overwriting recorded fixtures is a destructive op, see below).

Progressive disclosure. Verdict → counterexample → full diffs, enforced by file layout (§2.4), not by a UI toggle a reader might miss. Nobody is one click away from 4,200 fixtures’ worth of diff noise by accident.

Confirmation for destructive ops. Two operations are destructive and both are audited, not just confirmed:

Fixture deletion (harness lab down --purge or manual fixture pruning) requires --confirm <fixture-set-id> (typing the ID back, not a bare -y) and writes an audit event — fixtures are evidence for a QSA/SOX-relevant claim; deleting them is a scope decision, not housekeeping.
Normalization-rule changes (editing harness.config.ts’s exclusion list) are auditable scope decisions per the context brief’s binding language — every change is a version-controlled diff with a required rationale field the CLI refuses to let ship empty (harness normalize add <field> --reason "<text>" — no flag, no rule). This is the direct mechanism behind the “add normalization rule” triage decision in §3.2: it isn’t a checkbox, it’s a commit with a reason.

Long-running ops: progress + resumability. harness lab up (Docker boot, potentially slow per initial-recommendation’s named “boot pain” risk) and harness record (batches of live-traffic capture) both emit incremental progress to stderr (never mixed into the machine-readable stdout used by --ci mode) and are resumable — a killed record run leaves partial fixtures tagged incomplete in the manifest and a re-run continues rather than restarting; a killed verify batch resumes from the last committed fixture comparison. Resumability matters specifically because 50-60+ fixture classes are expected for Orders alone (context brief §D4 digest) — a non-resumable batch job at that scale is a standing invitation to lose an afternoon to a flaky Docker boot.

Error prevention, structural not disciplinary. Restated because it is the single most important Norman-lens decision in this document: a unit cannot reach the PR lane without a PARITY + selfCheckPassed=true work-ledger row (§2.5). This is a database-level gate on the promotion query, not a CLI warning someone can --force past. If a --force-equivalent is ever needed for a genuine program-level override (Tomas overriding a stuck gate), that override itself must write its own audited work-ledger row (verdict: FORCED, distinct value, never disguised as PARITY) — out of MVP scope unless a real deadlock surfaces; noted here so nobody quietly adds a silent bypass flag later.

AI-content behavior. An LLM may cluster/explain divergences (triage assist) but is structurally prevented from writing a verdict field — the schema’s verdict enum is written only by the deterministic diff engine; any LLM-generated text is confined to an advisory sub-object ({"advisory": {"llmSummary": "...", "clusteredWith": [...], "model": "..."}}) that the work-ledger promotion query never reads. Every rendered surface (bundle, CI annotation, triage view) that includes LLM text labels it inline as “advisory — not verdict authority”, not as a badge that can be scrolled past. This is directly downstream of D6 Q6: the 2024-2026 literature on LLM-as-judge for code/behavior equivalence is young and directionally supports triage-not-gate, with no paper found recommending LLM judges as a sole/final equivalence authority — the schema enforces what the evidence already counsels.

5. Visual direction

Scale to what matters — the evidence bundle’s markdown rendering is the only surface that gets real typographic attention; everything else is terminal output or a JSON blob with no visual design question at all. [SCALED DOWN]: no design system, no component library, no brand palette — this product has no logo.

Evidence-bundle typography/layout — scannable in 30 seconds:

counterexample.md fits one terminal screen (≈24 lines) or one un-scrolled markdown-render pane: verdict line, the single field that diverged, legacy value vs candidate value side by side (a two-column table, not prose), replay-recipe one-liner at the bottom.
Headings signal hierarchy, not decoration: # Verdict → ## Counterexample → ## Normalization ledger → ## Replay recipe → ## Full diffs (N files, not shown). The last heading is always present even when N=0 — consistency of structure across PARITY and DIVERGED bundles is itself a trust signal (a reader learns the shape once, trusts it every time).
Numbers carry their tag inline, per Zaruba’s evidence standard: parityRate: 0.99987 (MEASURED, 4200 fixtures, 2026-07-18) — never a bare percentage.

Diff presentation conventions:

Field-level diffs render as legacy → candidate (arrow, not a two-line unified diff) for scalar fields — this is a structured-data diff problem, not a text-diff problem, and should not borrow line-based diff conventions (+/-) that imply the wrong mental model.
Structural (nested JSON) diffs use a collapsed path notation (response.body.lineItems[2].totalCents: 4899 → 4901) rather than a pretty-printed nested-object diff — one line per actual divergence, no scrolling through unchanged sibling fields.
Normalized-away fields are never silently absent from a diff render — if a field would have differed but was excluded by a normalization rule, the ledger entry for that rule is cross-referenced inline (createdAt: excluded — see normalization-ledger.md#ts.createdAt), so a skeptical reader can see exactly what was hidden and why, not just that something was.

No dashboards beyond the one panel (§2.6). This is a Rams-lens decision as much as a scope decision: every additional panel is a claim of importance the program hasn’t earned the right to make yet, and dashboard sprawl is explicitly named as a UX risk (§6).

6. Prototype recommendation + UX risks

Prototype recommendation: build the evidence-bundle markdown renderer and the verdict JSON schema before the recorder or lab — literally hand-write one fixture set’s worth of fake diff data and render the bundle first. The report is the product’s face (initial-recommendation §7); if the 30-second-scan claim in §5 doesn’t hold on a hand-authored example, no amount of recorder/lab engineering fixes it later. This is cheap (hours, not days) and de-risks the highest-consequence design bet before the expensive plumbing (legacy lab, MBUS shadow-taps) gets built underneath it.

UX risks, ranked:

Noise → trust collapse → bypass (top risk, inherited and re-affirmed). If timestamps, generated IDs, ordering, or float formatting drown the signal, agents burn tokens on phantom DIVERGED verdicts, humans stop opening bundles, and the fleet routes around the harness at 99.999%-stakes moments — the exact failure mode named in initial-recommendation §10. Mitigation, UX-specific: the self-check (§1) must run on every verify, not just at bring-up, and a self-check failure must be louder in every rendered surface than a normal DIVERGED (distinct exit code 4, distinct CI glyph consideration) — a self-check failure means “don’t trust this tool right now,” which is a categorically different message than “this unit has a bug.”
Normalization-rule sprawl. Every DIVERGED-turned-acceptable case tempts a quick rule addition; if the ledger grows into hundreds of ad hoc exclusions with thin rationale, it stops being a trust artifact and becomes exactly the kind of hidden-exclusion-list the design explicitly rejects in §1. Mitigation, UX-specific: the mandatory --reason text (§4) is necessary but not sufficient — recommend a periodic (per-flagship-gate, not continuous) human review pass over the normalization ledger’s total rule count as a DoD-adjacent check, flagged here as a process recommendation outside this document’s UI scope.
Bundle overload. 50-60+ fixture classes for Orders alone (context brief §D4 digest, MODELED) risk evidence bundles that are individually scannable but collectively overwhelming when a builder is triaging several at once. Mitigation, UX-specific: the triage view (§3.3) exists precisely to be the dedup/routing layer above individual bundles — its one-line-per-divergence design is the pressure-release valve; if triage volume grows past what one line each can convey, that is itself a signal the harness has outgrown Orders-adapter scope and needs the flagship-gate re-scoping the initial recommendation already anticipates (assumption #1, “harness doesn’t generalize past Orders”).
CLI flag creep. Five verbs (§2.1) is the whole surface today; every additional flag is a small individually-justifiable erosion of the “one obvious command” Norman-lens goal from initial-recommendation §7. Mitigation, UX-specific: no flag ships without removing or subsuming an existing one, or without a documented unit-economics case tied to a flagship gate — this document is the enforcement point; a future editor of this file should treat a growing flag table as a finding to report, not a fact to accept.

Open questions

Does the 99.999% ledger-reconciliation DONE-row require a live-shadow “screen”/artifact this document hasn’t designed? D6 Q4 found no real-world precedent (Uber, Stripe) for recorded-fixture replay alone clearing a financial-grade cutover gate — every credible example used live/continuous mechanisms. If the program resolves that open question (context brief / initial-recommendation assumption #2) toward “yes, live dual-run is needed for the final gate,” this spec’s artifact set is incomplete — a live-shadow status view/dashboard would need its own IA pass, not a retrofit of the recorded-replay bundle design above. Recommend re-opening this document if/when that question resolves toward live-shadow.
UNVERIFIABLE reason taxonomy (§3.4) is a first draft, not field-tested. It’s built from the risks this research surfaced (lab boot, MBUS-tap gap, DARK batch jobs) but no real harness lab up run has ever produced a failure to validate the taxonomy’s completeness or wording against. Expect it to grow or get renamed after the first pilot adapter’s real UNVERIFIABLE runs.
Whether the CI annotation surface (§3.5) integrates with an existing Groupon CI check-list UI or needs its own rendering path is unresolved — this document assumes a generic PR-check annotation capability exists but did not verify which CI system / annotation API the monorepo actually uses.
Diffy’s unpublished false-positive rate (D1, D6) means this harness’s own self-check pass rate over its first real weeks of use is the only evidence anyone will ever have for whether the normalization approach in §1/§6 actually works — worth deciding now (outside this document) what “acceptable self-check failure rate during bring-up” looks like, so the first pilot doesn’t quietly redefine trust downward under schedule pressure.