e2e-test-scenarios

Instructions

This skill provides comprehensive end-to-end testing capabilities for Supabase applications, covering database operations, authentication flows, AI features (pgvector), realtime subscriptions, and production readiness validation.

Phase 1: Setup Test Environment

1. Initialize test environment:

   bash scripts/setup-test-env.sh
   

   This creates:

   • Test database configuration
   • Environment variables for testing
   • Test data fixtures
   • CI/CD configuration templates

2. Configure test database:

   • Use dedicated test project or local Supabase instance
   • Never run tests against production
   • Set SUPABASE_TEST_URL and SUPABASE_TEST_ANON_KEY
   • Enable pgTAP extension for database tests

3. Install dependencies:

   npm install --save-dev @supabase/supabase-js jest @types/jest
   # or
   pnpm add -D @supabase/supabase-js vitest
   

Phase 2: Database Workflow Testing

Test complete database operations from schema to queries:

1. Run database E2E tests:

   bash scripts/test-database-workflow.sh
   

   This validates:

   • Schema creation and migrations
   • Table relationships and foreign keys
   • RLS policies enforcement
   • Triggers and functions
   • Data integrity constraints

2. Use pgTAP for SQL-level tests:

   # Run all database tests
   supabase test db
   
   # Run specific test file
   supabase test db --file tests/database/users.test.sql
   

3. Test schema migrations:

   # Test migration up/down
   supabase db reset --linked
   supabase db push
   
   # Verify schema state
   bash scripts/validate-schema.sh
   

Phase 3: Authentication Flow Testing

Test complete auth workflows end-to-end:

1. Run authentication E2E tests:

   bash scripts/test-auth-workflow.sh
   

   This validates:

   • Email/password signup and login
   • Magic link authentication
   • OAuth provider flows (Google, GitHub, etc.)
   • Session management and refresh
   • Password reset flows
   • Email verification
   • Multi-factor authentication (MFA)

2. Test RLS with authenticated users:

   // See templates/auth-tests.ts for complete examples
   test('authenticated users see only their data', async () => {
     const { data, error } = await supabase
       .from('private_notes')
       .select('*');
   
     expect(data).toHaveLength(userNoteCount);
     expect(data.every(note => note.user_id === userId)).toBe(true);
   });
   

3. Test session persistence:

   # Run session validation tests
   npm test -- auth-session.test.ts
   

Phase 4: AI Features Testing (pgvector)

Test vector search and semantic operations:

1. Run AI features E2E tests:

   bash scripts/test-ai-features.sh
   

   This validates:

   • pgvector extension is enabled
   • Embedding storage and retrieval
   • Vector similarity search accuracy
   • HNSW/IVFFlat index performance
   • Hybrid search (keyword + semantic)
   • Embedding dimension consistency

2. Test vector search accuracy:

   // See templates/vector-search-tests.ts
   test('semantic search returns relevant results', async () => {
     const queryEmbedding = await generateEmbedding('machine learning');
   
     const { data } = await supabase.rpc('match_documents', {
       query_embedding: queryEmbedding
       match_threshold: 0.7
       match_count: 5
     });
   
     expect(data.length).toBeGreaterThan(0);
     expect(data[0].similarity).toBeGreaterThan(0.7);
   });
   

3. Test embedding operations:

   # Validate embedding pipeline
   npm test -- embedding-workflow.test.ts
   

4. Performance benchmarking:

   # Run performance tests
   bash scripts/benchmark-vector-search.sh [TABLE_NAME] [VECTOR_DIM]
   

Phase 5: Realtime Features Testing

Test realtime subscriptions and presence:

1. Run realtime E2E tests:

   bash scripts/test-realtime-workflow.sh
   

   This validates:

   • Database change subscriptions
   • Broadcast messaging
   • Presence tracking
   • Connection handling
   • Reconnection logic
   • Subscription cleanup

2. Test database change subscriptions:

   // See templates/realtime-tests.ts
   test('receives real-time updates on insert', async () => {
     const updates = [];
   
     const subscription = supabase
       .channel('test-channel')
       .on('postgres_changes'
         { event: 'INSERT', schema: 'public', table: 'messages' }
         (payload) => updates.push(payload)
       )
       .subscribe();
   
     await supabase.from('messages').insert({ content: 'test' });
   
     await waitFor(() => expect(updates).toHaveLength(1));
   });
   

3. Test presence and broadcast:

   # Run presence tests
   npm test -- presence.test.ts
   

Phase 6: Complete Integration Tests

Run full workflow tests simulating real user scenarios:

1. Execute complete E2E test suite:

   bash scripts/run-e2e-tests.sh
   

   This runs:

   • User signup → profile creation → data CRUD → logout
   • Document upload → embedding generation → semantic search
   • Chat message → realtime delivery → read receipts
   • Multi-user collaboration scenarios
   • Error handling and recovery

2. Run test scenarios in parallel:

   # Run all test suites
   npm test -- --maxWorkers=4
   
   # Run specific workflow
   npm test -- workflows/document-rag.test.ts
   

3. Generate test reports:

   # Generate coverage report
   npm test -- --coverage
   
   # Generate HTML report
   npm test -- --coverage --coverageReporters=html
   

Phase 7: CI/CD Integration

Setup automated testing in CI/CD pipelines:

1. Use GitHub Actions template:

   # Copy CI config to your repo
   cp templates/ci-config.yml .github/workflows/supabase-tests.yml
   

2. Configure test secrets:

   • Add SUPABASE_TEST_URL to GitHub secrets
   • Add SUPABASE_TEST_ANON_KEY to GitHub secrets
   • Add SUPABASE_TEST_SERVICE_ROLE_KEY for admin tests

3. Run tests on pull requests:

   • Automatic test execution on PR creation
   • Blocking PRs if tests fail
   • Test result reporting in PR comments

Phase 8: Cleanup and Teardown

Clean up test resources after testing:

1. Run cleanup script:

   bash scripts/cleanup-test-resources.sh
   

   This removes:

   • Test users and sessions
   • Test data from tables
   • Temporary test databases
   • Test file uploads

2. Reset test database:

   # Reset to clean state
   supabase db reset --linked
   
   # Or use migration-based reset
   bash scripts/reset-test-db.sh
   

Test Coverage Areas

Database Testing

• Schema Validation: Table structure, columns, constraints
• Migration Testing: Up/down migrations, rollback safety
• RLS Policies: Access control, policy enforcement
• Functions & Triggers: Database logic, event handlers
• Performance: Query optimization, index usage

Authentication Testing

• User Flows: Signup, login, logout, password reset
• Provider Integration: OAuth, magic links, phone auth
• Session Management: Token refresh, expiration, persistence
• MFA: Setup, verification, recovery codes
• Security: Rate limiting, brute force protection

AI Features Testing

• Vector Operations: Insert, update, delete embeddings
• Similarity Search: Accuracy, relevance, threshold tuning
• Index Performance: HNSW vs IVFFlat, query speed
• Hybrid Search: Combined keyword and semantic search
• Dimension Validation: Embedding size consistency

Realtime Testing

• Subscriptions: Database changes, broadcasts, presence
• Connection Management: Connect, disconnect, reconnect
• Message Delivery: Ordering, reliability, deduplication
• Performance: Latency, throughput, concurrent users
• Cleanup: Subscription disposal, memory leaks

Integration Testing

• Multi-Component: Auth + Database + Storage
• User Journeys: Complete workflows end-to-end
• Error Scenarios: Network failures, invalid data, rate limits
• Edge Cases: Concurrent updates, race conditions, timeouts

Test Data Strategies

Fixture Management

// Load test fixtures
const testData = await loadFixtures('users', 'posts', 'comments');

// Seed database
await seedDatabase(testData);

// Cleanup after tests
afterAll(async () => {
  await cleanupFixtures();
});

Factory Patterns

// Create test users with factories
const user = await createTestUser({
  email: 'test@example.com'
  metadata: { role: 'admin' }
});

// Create related data
const posts = await createTestPosts(user.id, 5);

Isolation Strategies

• Test Database: Dedicated test project
• Transaction Rollback: Rollback after each test
• Namespace Prefixes: test_ prefix for test data
• Time-based Cleanup: Delete data older than X hours

Performance Benchmarks

Query Performance Targets

• Simple queries: < 50ms
• Vector similarity search: < 100ms
• Hybrid search: < 200ms
• Complex joins: < 300ms

Realtime Latency Targets

• Message delivery: < 100ms
• Presence updates: < 200ms
• Database changes: < 500ms

Throughput Targets

• Authenticated requests: 100+ req/s
• Vector searches: 50+ req/s
• Realtime messages: 1000+ msg/s

Common Test Patterns

Pattern 1: Auth-Protected Resource Test

test('authenticated user CRUD operations', async () => {
  // 1. Sign up user
  const { user } = await supabase.auth.signUp({
    email: 'test@example.com'
    password: 'test123'
  });

  // 2. Create resource
  const { data } = await supabase
    .from('notes')
    .insert({ content: 'test note' })
    .select()
    .single();

  // 3. Verify ownership
  expect(data.user_id).toBe(user.id);

  // 4. Update resource
  await supabase
    .from('notes')
    .update({ content: 'updated' })
    .eq('id', data.id);

  // 5. Delete resource
  await supabase.from('notes').delete().eq('id', data.id);
});

Pattern 2: Vector Search Workflow Test

test('document RAG workflow', async () => {
  // 1. Upload document
  const doc = await uploadDocument('test.pdf');

  // 2. Generate embeddings
  const chunks = await chunkDocument(doc);
  const embeddings = await generateEmbeddings(chunks);

  // 3. Store in database
  await supabase.from('document_chunks').insert(
    chunks.map((chunk, i) => ({
      content: chunk
      embedding: embeddings[i]
      document_id: doc.id
    }))
  );

  // 4. Perform semantic search
  const query = 'What is the main topic?';
  const queryEmbedding = await generateEmbedding(query);

  const { data } = await supabase.rpc('match_documents', {
    query_embedding: queryEmbedding
    match_count: 5
  });

  // 5. Verify results
  expect(data.length).toBeGreaterThan(0);
  expect(data[0].similarity).toBeGreaterThan(0.7);
});

Pattern 3: Realtime Collaboration Test

test('multi-user realtime chat', async () => {
  // 1. Create two users
  const user1 = await createTestUser();
  const user2 = await createTestUser();

  // 2. Subscribe to messages
  const user1Messages = [];
  const user2Messages = [];

  const sub1 = await subscribeToMessages(user1, user1Messages);
  const sub2 = await subscribeToMessages(user2, user2Messages);

  // 3. User 1 sends message
  await sendMessage(user1, 'Hello from user 1');

  // 4. Verify both users receive it
  await waitFor(() => {
    expect(user1Messages).toHaveLength(1);
    expect(user2Messages).toHaveLength(1);
  });

  // 5. Cleanup subscriptions
  await sub1.unsubscribe();
  await sub2.unsubscribe();
});

Troubleshooting

Tests Failing Intermittently

• Race Conditions: Add proper wait conditions
• Async Issues: Ensure all promises are awaited
• Cleanup: Verify test isolation and cleanup
• Timeouts: Increase timeout for slow operations

Tests Passing Locally but Failing in CI

• Environment: Verify env variables in CI
• Timing: CI may be slower, increase timeouts
• Parallelization: Check for shared state issues
• Dependencies: Ensure all deps are installed

Slow Test Execution

• Parallel Tests: Use --maxWorkers flag
• Test Selection: Run only changed tests
• Database Reset: Use faster cleanup methods
• Mocking: Mock external services

Database Connection Issues

• Connection Pooling: Limit concurrent connections
• Cleanup: Properly close connections after tests
• Retries: Implement connection retry logic
• Timeouts: Set appropriate connection timeouts

Files Reference

Scripts:

• scripts/setup-test-env.sh - Initialize test environment
• scripts/run-e2e-tests.sh - Execute complete test suite
• scripts/test-database-workflow.sh - Database E2E tests
• scripts/test-auth-workflow.sh - Authentication E2E tests
• scripts/test-ai-features.sh - Vector search E2E tests
• scripts/test-realtime-workflow.sh - Realtime E2E tests
• scripts/cleanup-test-resources.sh - Clean up test data
• scripts/validate-schema.sh - Validate database schema
• scripts/benchmark-vector-search.sh - Performance benchmarks
• scripts/reset-test-db.sh - Reset test database

Templates:

• templates/test-suite-template.ts - Jest/Vitest boilerplate
• templates/database-tests.ts - Database operation tests
• templates/auth-tests.ts - Authentication flow tests
• templates/vector-search-tests.ts - pgvector tests
• templates/realtime-tests.ts - Realtime subscription tests
• templates/ci-config.yml - GitHub Actions CI/CD
• templates/jest.config.js - Jest configuration
• templates/vitest.config.ts - Vitest configuration

Examples:

• examples/complete-test-workflow.md - Full E2E testing guide
• examples/ci-cd-integration.md - CI/CD setup guide
• examples/test-data-strategies.md - Test data management
• examples/performance-benchmarks.md - Performance testing
• examples/mocking-strategies.md - Mocking external services

---

Plugin: supabase Version: 1.0.0 Last Updated: 2025-10-26