skills/stablyai/agent-skills/playwright-test-data-isolation

playwright-test-data-isolation

SKILL.md

Playwright Test Data Isolation

Use this skill when teams run Playwright tests in a shared QA/staging environment with:

  • A shared database (often production-connected)
  • Shared test user accounts
  • Parallel CI and local test runs
  • Flaky tests caused by data collisions, cross-test interference, or unsafe cleanup

Goal: make tests parallel-safe and operationally safe while staying standard Playwright.

When To Use This Skill

Use this skill for tasks like:

  • "Our Playwright tests share a database and keep colliding"
  • "How do we run E2E safely with shared accounts?"
  • "How should we isolate test data in CI?"
  • "How do we clean up test data without deleting real data?"
  • "How do we keep parallel Playwright runs stable?"
  • "How do we design fixtures for safe create-and-cleanup?"

Core Contract (Non-Negotiable)

Every test must follow this contract:

  1. Create the entities it needs (test-owned data).
  2. Mutate only entities it created.
  3. Treat shared baselines (accounts, workspaces, org settings) as read-only.
  4. Clean up by exact IDs captured during the test (never broad filters).

If a test cannot follow this contract, move it to a serial suite.

Non-Negotiable Guardrails

  • Treat shared users and shared workspaces as read-only containers.
  • Never update shared workspace settings, membership, billing, or global toggles in parallel suites.
  • Prefix test-created records with a unique namespace (e2e-<run>-<worker>-<uuid>).
  • Delete by exact IDs; never run broad delete filters in test teardown.
  • Keep a scheduled stale-data janitor as backup, not as the primary cleanup mechanism.

Recommended Architecture

Use shared accounts for authentication only, then isolate all mutable data by namespace.

  • Shared accounts: identity only
  • Test data: fully owned by one test via namespace
  • Cleanup: deterministic, ID-based, reverse order
  • Shared-state mutation suites: serialized
  • Scheduled cleanup: removes stale e2e-* artifacts older than a threshold

Account Strategy

Pick the highest isolation level your environment can support:

  1. Best: one account per worker (pool accounts and map by workerIndex).
  2. Acceptable: shared account for sign-in only, test-owned child data for all writes.
  3. Avoid: many tests mutating top-level account/workspace settings in parallel.

Implementation Pattern (Playwright-Native)

1) Reuse auth state for shared accounts (setup project)

// auth/setup-auth.ts
import { test as setup } from "@playwright/test";

setup("login as shared qa account", async ({ page }) => {
  await page.goto(process.env.BASE_URL!);
  await page.getByLabel("Email").fill(process.env.E2E_USER_EMAIL!);
  await page.getByLabel("Password").fill(process.env.E2E_USER_PASSWORD!);
  await page.getByRole("button", { name: "Sign in" }).click();
  await page.context().storageState({ path: "playwright/.auth/qa-user.json" });
});

Wire this into a setup project dependency in playwright.config.ts so parallel projects reuse a stable signed-in state.

2) Add namespace + cleanup tracker fixtures

// tests/fixtures/test-data.ts
import { test as base } from "@playwright/test";
import { randomUUID } from "crypto";

type CleanupFn = () => Promise<void>;

type IsolationFixtures = {
  namespace: string;
  trackCleanup: (fn: CleanupFn) => void;
};

export const test = base.extend<IsolationFixtures>({
  namespace: async ({}, use, testInfo) => {
    const runId = process.env.CI_RUN_ID ?? process.env.GITHUB_RUN_ID ?? "local";
    const ns = [
      "e2e",
      runId,
      testInfo.project.name,
      String(testInfo.workerIndex),
      randomUUID(),
    ].join("-");
    await use(ns);
  },

  trackCleanup: async ({}, use) => {
    const fns: CleanupFn[] = [];
    await use((fn: CleanupFn) => fns.push(fn));

    for (const fn of fns.reverse()) {
      try {
        await fn();
      } catch (err) {
        console.error("cleanup failed", err);
      }
    }
  },
});

export { expect } from "@playwright/test";

Notes:

  • runId + project + worker + uuid keeps names unique across CI, shards, and local runs.
  • Reverse-order cleanup (LIFO) avoids dependency-order teardown bugs.

3) Create child resources in shared containers only

// tests/e2e/projects.spec.ts
import { test, expect } from "../fixtures/test-data";

test.use({ storageState: "playwright/.auth/qa-user.json" });

test("creates isolated project safely", async ({ page, request, namespace, trackCleanup }) => {
  const workspaceId = process.env.E2E_SHARED_WORKSPACE_ID!; // read-only container
  const projectName = `e2e-${namespace}`;

  await page.goto(`/workspaces/${workspaceId}/projects`);
  await page.getByRole("button", { name: "New project" }).click();
  await page.getByLabel("Project name").fill(projectName);
  await page.getByRole("button", { name: "Create" }).click();

  const projectId = await getProjectIdByName(request, workspaceId, projectName);
  trackCleanup(async () => {
    await deleteProjectById(request, workspaceId, projectId);
  });

  await expect(page.getByText(projectName)).toBeVisible();
});

async function getProjectIdByName(
  request: import("@playwright/test").APIRequestContext,
  workspaceId: string,
  name: string,
) {
  const res = await request.get(
    `/api/internal/workspaces/${workspaceId}/projects?name=${encodeURIComponent(name)}`,
  );
  if (!res.ok()) throw new Error(`project lookup failed: ${res.status()}`);
  const body = await res.json();
  return body.items[0].id as string;
}

async function deleteProjectById(
  request: import("@playwright/test").APIRequestContext,
  workspaceId: string,
  id: string,
) {
  const res = await request.delete(`/api/internal/workspaces/${workspaceId}/projects/${id}`);
  if (!res.ok()) throw new Error(`project delete failed: ${res.status()}`);
}

4) Isolate true shared-state tests in serial suites

import { test } from "@playwright/test";

test.describe.configure({ mode: "serial" });

Only use this for tests that must mutate global/shared settings. Keep most tests fully parallel by using test-owned child resources.

5) Add a stale-data janitor project (backup safety net)

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  projects: [
    {
      name: "e2e",
      testDir: "tests/e2e",
      use: { storageState: "playwright/.auth/qa-user.json" },
    },
    {
      name: "cleanup",
      testDir: "tests/cleanup",
      workers: 1,
      retries: 0,
    },
  ],
});
// tests/cleanup/stale-e2e-data.spec.ts
import { test, expect } from "@playwright/test";

test("delete stale e2e data older than 24h", async ({ request }) => {
  const res = await request.post("/api/internal/cleanup/e2e", {
    data: {
      prefix: "e2e-",
      olderThanHours: 24,
    },
  });

  expect(res.ok()).toBeTruthy();
});

Run janitor on a schedule (nightly or multiple times/day), but keep per-test cleanup as the primary defense.

Collision Debugging Checklist

When a test collides, collect these first:

  • Namespace used by each failing test
  • Worker index and project name
  • IDs created by each test before cleanup
  • Which shared entity was mutated (if any)

If retries make failures disappear, treat that as a signal of shared-state coupling, not success.

Optional: If Team Uses Stably

Stably can help with orchestration and operations, but this strategy does not depend on it.

  • Use npx stably test as a Playwright-compatible runner wrapper when you want hosted reporting/ops.
  • Use Stably environments to inject shared account credentials by environment.
  • Use Stably schedules/cloud workers for janitor jobs and large parallel runs.
  • Keep the same isolation contract regardless of runner.

Anti-Patterns To Block

  • Parallel tests writing to the same shared entity
  • Shared mutable fixtures reused across files
  • Cleanup using broad filters like "delete all e2e rows"
  • Relying on execution order
  • Using retries to mask shared-data collisions
  • Treating nightly cleanup as the only cleanup

Agent Output Requirements

When this skill is activated, the agent should:

  1. Confirm environment constraints (shared DB, shared accounts, parallelism level).
  2. Produce or update a namespace fixture and deterministic cleanup tracker.
  3. Classify tests into parallel-safe vs shared-state-mutation suites.
  4. Refactor at least one target test to create-and-own data.
  5. Add or propose a stale-data cleanup project and schedule.
  6. Return a short risk report listing remaining unsafe tests.

Rollout Plan

  1. Add namespace + cleanup fixtures.
  2. Migrate flaky mutable tests to create-and-own data.
  3. Add cleanup project and schedule.
  4. Move shared-state mutation tests to serial mode.
  5. Increase workers only after collision-free runs.

Success Criteria

  • Parallel runs no longer collide on shared data.
  • Teardown never touches non-test data.
  • Flake rate drops for data-dependent tests.
  • Shared environment remains stable for manual QA and CI.
Weekly Installs
1
GitHub Stars
4
First Seen
12 days ago
Installed on
amp1
cline1
opencode1
cursor1
kimi-cli1
codex1