Generating E2E Tests

This skill generates complete end-to-end test scenarios from user journey definitions, requirements, or by analyzing the application's navigation structure.

When to Use

• You need to create E2E tests for critical user flows (login, checkout, signup)
• You're building a regression suite for CI/CD
• You have user stories or journey maps to convert to automated tests
• You want to generate Playwright test code from natural language descriptions

Workflow

Step 1 — Define User Journeys

Journeys can come from three sources:

A) Provided by user: Accept JSON journey definitions (see format below)

B) Generated from requirements: Parse Jira tickets or user stories:

python skills/qa-generating-e2e-tests/scripts/journey_from_requirements.py \
  --requirements requirements.json \
  --output journeys.json

C) Discovered from the app: Navigate the application and infer critical paths:

python skills/qa-generating-e2e-tests/scripts/discover_journeys.py \
  --url https://staging.example.com \
  --output journeys.json

Step 2 — Generate Test Code

Convert journey definitions into executable Playwright tests:

python skills/qa-generating-e2e-tests/scripts/generate_playwright.py \
  --journeys journeys.json \
  --output tests/ \
  --base-url https://staging.example.com

This produces one test file per journey, using Playwright's async API with:

• Semantic element selection (by role, text, label — not brittle selectors)
• Automatic screenshot capture at each step
• Console error collection
• Network failure monitoring
• Configurable timeouts and retry logic

Step 3 — Execute Tests

python skills/qa-generating-e2e-tests/scripts/run_e2e.py \
  --tests tests/ \
  --base-url https://staging.example.com \
  --output results/ \
  --browsers chromium

Journey Definition Format

{
  "journey_id": "user-signup",
  "name": "New User Signup Flow",
  "priority": "critical",
  "preconditions": [],
  "steps": [
    {
      "step": 1,
      "action": "navigate",
      "target": "/signup",
      "description": "Go to the signup page"
    },
    {
      "step": 2,
      "action": "fill",
      "target": "Email input",
      "value": "newuser@example.com",
      "description": "Enter email address"
    },
    {
      "step": 3,
      "action": "fill",
      "target": "Password input",
      "value": "SecurePass123!",
      "description": "Enter password"
    },
    {
      "step": 4,
      "action": "click",
      "target": "Sign Up button",
      "description": "Submit the signup form"
    },
    {
      "step": 5,
      "action": "wait",
      "target": "Welcome message or dashboard",
      "description": "Verify successful signup"
    }
  ],
  "postconditions": [
    "User account exists in the system",
    "Welcome email sent"
  ]
}

Action types: navigate, click, fill, select, hover, scroll, wait, assert_visible, assert_text, assert_url, screenshot

Target format: Use human-readable descriptions. The test generator converts these to Playwright locators using get_by_role(), get_by_text(), get_by_label(), and get_by_placeholder() — avoiding CSS selectors or XPaths.

Generated Test Structure

import pytest
from playwright.async_api import async_playwright, expect

@pytest.mark.asyncio
async def test_user_signup():
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        context = await browser.new_context(viewport={"width": 1440, "height": 900})
        page = await context.new_page()

        # Step 1: Go to the signup page
        await page.goto("https://staging.example.com/signup")
        await page.screenshot(path="screenshots/signup-01-page-loaded.png")

        # Step 2: Enter email address
        await page.get_by_label("Email").fill("newuser@example.com")

        # Step 3: Enter password
        await page.get_by_label("Password").fill("SecurePass123!")

        # Step 4: Submit the signup form
        await page.get_by_role("button", name="Sign Up").click()

        # Step 5: Verify successful signup
        await expect(page.get_by_text("Welcome")).to_be_visible(timeout=10000)
        await page.screenshot(path="screenshots/signup-05-success.png")

        await browser.close()

Test Data Management

The skill includes helpers for test data lifecycle:

• Before test: Create users, seed data via API
• After test: Clean up created data
• Shared fixtures: Reusable login sessions, pre-populated carts

# Generated fixture example
@pytest.fixture
async def authenticated_page(page):
    """Login and return an authenticated page."""
    await page.goto(f"{BASE_URL}/login")
    await page.get_by_label("Email").fill(TEST_USER_EMAIL)
    await page.get_by_label("Password").fill(TEST_USER_PASSWORD)
    await page.get_by_role("button", name="Log in").click()
    await page.wait_for_url("**/dashboard")
    yield page

Handling Dynamic Content

The generated tests handle dynamic values without hardcoded waits:

• Use wait_for_selector instead of time.sleep
• Use expect(locator).to_have_text(re.compile(...)) for dynamic text
• Extract and reuse generated IDs (order numbers, confirmation codes)
• Use page.wait_for_response() for API-dependent UI updates