Generating Integration Tests

This skill turns API contracts into comprehensive test suites that validate service integrity beyond basic schema conformance.

When to Use

• You have an OpenAPI spec, GraphQL schema, or list of API endpoints
• You need to generate test cases that exercise business logic, not just structure
• You want to validate that services handle edge cases, errors, and adversarial inputs
• You're testing cross-service data pipelines for consistency

Workflow

Step 1 — Parse the Contract

Read the API specification and extract a structured inventory:

python skills/qa-generating-integration-tests/scripts/parse_contract.py \
  --input <openapi.yaml|schema.graphql|endpoints.json> \
  --output contract-inventory.json

The inventory contains, for each endpoint:

• Method, path, description
• Request parameters (path, query, header, body)
• Request body schema with types and constraints
• Response schemas per status code
• Authentication requirements
• Rate limits (if documented)

Step 2 — Generate Test Cases

For each endpoint, generate tests across these categories:

Positive tests (happy path):

• Valid request with all required fields → expected success response
• Valid request with optional fields included → expected success response
• Valid request with minimum required fields → expected success response

Negative tests (error handling):

• Missing required fields → 400 with descriptive error
• Invalid field types (string where int expected) → 400
• Invalid field values (negative quantity) → 400 or 422
• Missing authentication → 401
• Insufficient permissions → 403
• Nonexistent resource → 404

Boundary tests:

• Minimum values for numeric fields
• Maximum values for numeric fields
• Empty strings for string fields
• Maximum length strings
• Empty arrays, single-element arrays, large arrays
• Null values for nullable fields
• Special characters in string fields

Security tests:

• SQL injection patterns in string fields
• XSS payloads in text fields
• Path traversal in file-related endpoints
• Authentication token manipulation

Use the generation script:

python skills/qa-generating-integration-tests/scripts/generate_tests.py \
  --contract contract-inventory.json \
  --output test-suite.json \
  --categories positive,negative,boundary,security

Step 3 — Execute Tests

Run the generated test suite against the target environment:

python skills/qa-generating-integration-tests/scripts/run_tests.py \
  --suite test-suite.json \
  --base-url https://staging.example.com \
  --auth-token $AUTH_TOKEN \
  --output results.json \
  --parallel 5

Step 4 — Analyze Results

Classify each result and generate the report:

• Pass: Response matches expectations
• Fail: Response differs from expectations (potential bug)
• Error: Request could not be sent (infrastructure issue)
• Unexpected Pass: Request that should have been rejected was accepted

Contract Formats

OpenAPI 3.x

Read references/openapi-parsing.md for detailed parsing rules. Key areas:

• paths → endpoints
• components/schemas → data types
• security → authentication schemes
• parameters → query/path/header params

GraphQL

Read references/graphql-parsing.md. Key areas:

• Queries and mutations → equivalent to REST endpoints
• Input types → request schemas
• Return types → response schemas
• Custom scalars → validation rules

Manual Endpoint List

Accept a JSON array of endpoint definitions when no formal spec exists:

[
  {
    "method": "POST",
    "path": "/api/users",
    "description": "Create a new user",
    "body": {"email": "string (required)", "name": "string (required)"},
    "responses": {"201": "User created", "400": "Validation error"}
  }
]

Test Case Format

Generated tests follow this structure:

{
  "test_id": "POST-users-001",
  "endpoint": "POST /api/users",
  "category": "positive",
  "description": "Create user with all required fields",
  "request": {
    "method": "POST",
    "path": "/api/users",
    "headers": {"Content-Type": "application/json"},
    "body": {"email": "test@example.com", "name": "Test User"}
  },
  "expected": {
    "status_code": 201,
    "body_contains": ["id", "email", "name"],
    "response_time_max_ms": 2000
  }
}

References

• references/openapi-parsing.md — OpenAPI 3.x parsing details
• references/graphql-parsing.md — GraphQL schema parsing details