Running User Acceptance Walkthroughs

Overview

Acceptance at delivery time should be experiential, not just a test summary. Primary goal: help the human directly see and feel what changed before merge. If work is not user-facing, run and show executable proof with user-impact translation. For user-facing work, launch the actual app and use the feature as a real user would — navigating screens, clicking buttons, filling forms, observing results. Then give the user instructions for running it themselves.

Hard Gates (Non-Negotiable)

Scope lock before any validation
- Start with a one-line scope statement:
  - UAT Scope: <ticket/PR scope only>
- If work drifts outside scope, stop and restate scope before continuing.
User-facing UAT requires a live walkthrough first
- Do not start with test suites.
- First demonstrate behavior in the running app (web/Electron/iOS/macOS) as a user would.
- Automated tests are supplementary evidence after the walkthrough.
No GO verdict without explicit human acceptance
- Before user confirmation, recommendation must be:
  - Recommendation: Pending user sign-off
- GO or GO with follow-ups is allowed only after explicit user acceptance (e.g. "accept", "approved", "looks good").
Do not claim acceptance completion unilaterally
- If the user has not accepted, acceptance is not complete.
- Do not frame CI/test pass as user acceptance.
Manual run guidance is mandatory for user-facing or mixed UAT
- After validation, always provide a Manual Run Instructions section.
- This is required even when:
  - the user did not ask for manual steps
  - automated tests already passed
  - screenshots/videos were already collected
- For user-facing or mixed mode, omission of manual run steps is a failed UAT response.
Manual instructions must be normal-user-path first
- Manual Run Instructions must describe the same path a product user would take in the app.
- Do not require test harnesses, seeded fixtures, or synthetic state setup as the primary path.
- Forbidden in primary manual steps:
  - playwright, vitest, npm test, CI commands
  - KATA_STATE_FILE=... or any fixture/seed file dependency
  - "inject", "seed", "mock event", "test-only flag"
- If normal path is genuinely blocked, provide:
  - Manual Run Instructions (Normal Path) first (best-effort UI path), then
  - Fallback (Engineering Only) second, clearly labeled as non-user path.

Demo means "use the app", not "run tests"

Running e2e tests is not a demo. Tests verify code correctness. A demo lets the user see the feature working in the real app, interactively, so they can judge whether the right thing was built.

For user-facing work, the sequence is always:

Start the app (dev server, simulator, Electron launch)
Navigate to the feature in the running app
Use it — click, type, scroll, interact as a user would
Show the result — screenshot or describe what appeared on screen

Only after the live demo, run tests if needed to confirm no regressions. Tests supplement the demo; they do not replace it.

If the user cannot watch live and explicitly asks for asynchronous evidence, state that this is evidence collection, not final acceptance, and keep recommendation as Pending user sign-off.

Output Contract (Must Follow)

For user-facing and mixed mode responses, include these sections in order:

UAT Scope: ...
Mode and Platform
Slice-by-slice result (Pass/Fail)
Evidence (screenshots/video and/or logs)
Manual Run Instructions (required; numbered user steps to run the feature manually)
Recommendation: Pending user sign-off (until explicit acceptance)
Please reply: accept / reject

Do not replace section 5 with Playwright/CI commands. Manual means in-app user interactions. Section 5 must be normal-user-path first and must not depend on state seed files or automated test tooling.

Pre-Response Self-Check (Required)

Before sending a user-facing/mixed UAT response, verify all are true:

Response contains Manual Run Instructions.
Instructions are UI-driven and executable by a normal user.
No forbidden automation/test/seed commands appear in the primary manual steps.
Steps include expected visible outcomes.

If any check fails, rewrite before sending.

When to Use

End of PR/ticket prompts: "UAT", "verify", "walk me through", "show what changed", "can we merge?"
Sign-off requests where confidence requires direct observation, not only CI output
Mixed work (UI + backend/infrastructure) that needs both walkthrough and proof

Do not use for mid-implementation debugging or code-quality review without acceptance intent.

Decision Flow

digraph uat_flow {
  "End-of-ticket or PR acceptance?" [shape=diamond];
  "Any user-facing behavior changed?" [shape=diamond];
  "Detect platform" [shape=diamond];
  "Can agent execute proof locally?" [shape=diamond];
  "Run web/Electron demo" [shape=box];
  "Run iOS/macOS demo" [shape=box];
  "Run CLI/API demo" [shape=box];
  "Provide reproducible proof plan + user run steps" [shape=box];

  "End-of-ticket or PR acceptance?" -> "Any user-facing behavior changed?" [label="yes"];
  "Any user-facing behavior changed?" -> "Detect platform" [label="yes"];
  "Any user-facing behavior changed?" -> "Can agent execute proof locally?" [label="no"];
  "Detect platform" -> "Run web/Electron demo" [label="web or Electron"];
  "Detect platform" -> "Run iOS/macOS demo" [label="iOS/macOS native"];
  "Detect platform" -> "Run CLI/API demo" [label="CLI/API/backend"];
  "Can agent execute proof locally?" -> "Run CLI/API demo" [label="yes"];
  "Can agent execute proof locally?" -> "Provide reproducible proof plan + user run steps" [label="no"];
}

Step-by-Step Workflow

1. Identify scope, mode, and platform

Confirm what behavior is being accepted.
Declare scope lock explicitly: UAT Scope: ....
Declare Mode: user-facing, non-user-facing, or mixed.
For user-facing work, detect Platform to select the right demo tool:

Signal	Platform	Playbook
Xcode project, `.xcodeproj`, `.swift` files, iOS simulator	iOS/macOS native	`./references/ios-demo-playbook.md`
Web app, `package.json` with dev server, browser-based UI	Web	`./references/web-demo-playbook.md`
Electron app, `electron` in dependencies	Electron	`./references/web-demo-playbook.md` (Electron section)
CLI tool, API endpoint, backend service, infrastructure	CLI/API	`./references/cli-api-demo-playbook.md`

If the platform is ambiguous, ask the user.

2. Define acceptance slices

Break validation into small slices (2-5), each with clear pass/fail criteria.

How to derive slices:

Start from the ticket's acceptance criteria if they exist
Otherwise, map each user-visible behavior change to a slice
For non-user-facing work, map each functional change to a demonstrable proof
Each slice should be independently verifiable — avoid slices that only pass if run in sequence

3. Execute validation

Follow the appropriate playbook:

User-facing (web/Electron): ./references/web-demo-playbook.md
User-facing (iOS/macOS): ./references/ios-demo-playbook.md
Non-user-facing or CLI/API: ./references/cli-api-demo-playbook.md
Mixed: run user-facing demo first, then technical proof tied to the same outcome.

For user-facing or mixed mode, include short narration of what is being done in-app while demonstrating (where you navigated, what you clicked, what changed on screen).

4. Capture evidence

Save screenshots/video for user-facing slices.
Save exact commands and key output lines for technical slices.

5. Report results to the user

Start with overview and scope bullets.
List each slice with explicit Pass/Fail.
Provide instructions to the user for running the same validation themselves.
For user-facing or mixed mode, include a dedicated Manual Run Instructions heading every time.
Manual steps must follow normal user path first; if fallback is needed, label it Fallback (Engineering Only).
End with one of:
- Recommendation: Pending user sign-off (default until explicit acceptance)
- Recommendation: GO
- Recommendation: GO with follow-ups
- Recommendation: NO-GO

Then ask for explicit acceptance decision:

Please reply: accept / reject

6. Update ticket (required)

Two-phase ticket updates are required:

Before user acceptance: post a status comment with Pending user sign-off and evidence.
After explicit user acceptance: post final verdict (GO / GO with follow-ups) and close acceptance.

If recommendation is GO or GO with follow-ups, post a ticket comment in the project system of record (check CLAUDE.md or project config; common systems: Linear, Jira, GitHub Issues) with:

UAT verdict (GO / GO with follow-ups)
Scope validated (2-5 bullets)
Mode and platform used
Slice-by-slice pass/fail summary
Evidence links or file paths (screenshots, videos, logs)
Commands run (for non-user-facing or mixed technical slices)
Any follow-ups or residual risks

If no ticket is known, ask for the ticket ID before closing UAT. Do not mark acceptance complete until this ticket update is posted.

Quick Reference

Mode	First step	Evidence required	Done when
user-facing	Run platform-appropriate live demo (slice 1)	Demo trace + screenshots/video + observed UI	User explicitly accepts/rejects
non-user-facing	Run proof command(s)	Command output + impact translation	Reproducible evidence reviewed
mixed	User-facing live demo first, then technical proof	Both demo evidence and technical proof	User explicitly accepts/rejects

Common Mistakes

Running e2e or UI tests instead of launching the app and using the feature
Omitting manual user-run instructions after reporting automated validation results
Using seeded state files or test-only setup as the primary manual path
Dumping a static checklist with no interaction
Reporting only test counts with no demonstration
Skipping a live demo for user-facing changes
Skipping non-UI demo because there is no frontend change
Declaring merge readiness before collecting explicit pass/fail signals
Declaring GO before explicit user acceptance
Using the wrong demo tool for the platform (e.g., agent-browser for an iOS app)

Rationalization Table

Excuse	Reality
"No UI changes, so UAT is just unit tests."	Non-user-facing work still needs demonstrable proof and user-impact explanation.
"We are in a rush, give a fast merge checklist."	Time pressure increases need for clear GO/NO-GO evidence.
"I already summarized everything; that is enough."	Summaries do not replace user experience or executable demonstration.
"User can test later after merge."	Acceptance belongs before merge unless explicitly deferred by user.
"Tests pass, so it works."	Tests prove code correctness. Acceptance proves the right thing was built.
"I'll run the e2e test suite as the demo."	E2e tests are automated assertions. A demo is using the app interactively.

Red Flags - Stop and Correct

You are about to run a test suite instead of launching the app and using the feature.
You are about to send a user-facing UAT result without a Manual Run Instructions section.
Your Manual Run Instructions include KATA_STATE_FILE, Playwright, or CI/test commands.
You are about to send only a summary/checklist.
You cannot point to any observed behavior or executed proof.
You are treating CI green status as equivalent to acceptance.
You are asking for merge without an explicit acceptance signal.
You skipped platform detection and defaulted to the wrong demo tool.

user-acceptance