Exploring Application UI

This skill drives autonomous exploration of web applications using visual AI to understand and interact with UI elements semantically — the way a human tester would see and navigate them.

When to Use

• No scripted tests exist yet and you need to assess application quality
• You want to discover "unknown unknown" bugs that scripted tests miss
• You're validating a new feature's integration with existing UI
• You need a quick health check of a staging environment

Navigation Strategy

Element Identification

Use visual analysis (screenshots + Claude Vision) to identify interactive elements. Do NOT rely on DOM selectors as the primary identification method.

Identification hierarchy:

1. Visible text labels ("Sign In", "Add to Cart")
2. Placeholder text ("Enter your email...")
3. Iconography (search icon, hamburger menu, close X)
4. Visual affordances (raised buttons, underlined links, input fields)
5. Spatial context (sidebar navigation, header menu, footer links)

State Tracking

Maintain a state machine as you explore:

exploration_state = {
    "visited_urls": set(),        # Pages we've been to
    "visited_states": set(),      # URL + key UI state (e.g., modal open, tab selected)
    "action_queue": [],           # Planned next actions, ordered by priority
    "dead_ends": [],              # States with no forward navigation
    "errors_seen": [],            # Collected error states
    "coverage_estimate": 0.0,    # Rough % of app explored
}

Exploration Priorities

When deciding what to explore next, use this priority order:

1. Unexplored navigation items — breadth first for top-level nav
2. Forms and inputs — fill with various inputs, submit, observe
3. Error states — intentionally trigger errors (empty submits, invalid data)
4. Edge navigation — back button, direct URL entry, bookmark behavior
5. Authentication boundaries — access control, session management
6. Responsive behavior — resize viewport, check layout integrity

Interaction Patterns

For each page, execute these interaction patterns:

Forms:

1. Submit completely empty → check error handling
2. Fill one field at a time → check progressive validation
3. Fill all fields correctly → check success path
4. Fill with edge-case values → check robustness
5. Submit, then back-button → check data persistence

Navigation:

1. Click every visible link/button → map the navigation graph
2. Right-click → check context menu behavior
3. Middle-click → check new-tab behavior
4. Keyboard navigation (Tab, Enter, Escape) → check accessibility
5. Browser back/forward → check history state management

Dynamic Content:

1. Scroll to bottom → check lazy loading
2. Wait 30 seconds → check for memory leaks or stale data
3. Toggle between tabs/views rapidly → check race conditions
4. Open multiple instances → check concurrent state

Bug Detection Heuristics

Use these rules to identify potential bugs during exploration:

Definite Bugs (always report)

• Page returns HTTP 4xx or 5xx
• JavaScript error in console
• Visible "undefined", "null", "NaN", or "[object Object]" in UI
• Form submits successfully with invalid/empty required data
• Click on element produces no visible response
• Page layout is visibly broken (overlapping elements, text cutoff)

Probable Bugs (report with lower severity)

• Loading spinner that never resolves (>10s)
• Flash of unstyled content (FOUC)
• Inconsistent button/link styling
• Missing favicon or broken images
• Placeholder text left in production ("Lorem ipsum", "TODO")
• Console warnings about deprecated APIs

UX Issues (report as info)

• Confusing navigation (>3 clicks to reach common actions)
• Missing confirmation dialogs for destructive actions
• Error messages that don't explain how to fix the issue
• No loading indicator during async operations
• Missing empty states (blank page when no data exists)

Using the Exploration Script

python skills/qa-exploring-application-ui/scripts/explore.py \
  --url https://staging.example.com \
  --duration 30 \
  --output exploration-results/ \
  --credentials user:pass

The script handles browser setup, screenshot capture, and state tracking. You drive the exploration decisions by analyzing each screenshot and choosing the next action.

Output Format

Produce a structured session log and a narrative summary. See the bug report skill (skills/qa-generating-bug-reports/SKILL.md) for bug formatting standards.

The narrative summary should be written for a human QA lead — cover what you explored, what you found, what areas you couldn't reach, and your overall assessment of the application's quality.