Test Data Management in Azure DevOps

Test data management is the unglamorous but critical work that determines whether your automated test suite is reliable or flaky. In Azure DevOps pipelines, it requires deliberate design: data must be available, isolated, and cleaned up.

---

The four test data problems in CI

1. Stale data: tests depend on records that were manually created and may change
2. Shared data conflicts: parallel tests read/write the same records
3. Leftover data: failed tests leave records that cause subsequent runs to fail
4. Sensitive data: production credentials or PII in test fixtures

---

Strategy 1: Database seeding in pipelines

Seed the test database at the start of each pipeline run:

YAML
1stages:
2  - stage: PrepareEnvironment
3    jobs:
4      - job: SeedDatabase
5        steps:
6          - script: npm ci
7          - script: npm run db:reset     # Drop and recreate test database
8            env:
9              DATABASE_URL: $(TEST_DB_URL)
10          - script: npm run db:migrate   # Apply schema migrations
11            env:
12              DATABASE_URL: $(TEST_DB_URL)
13          - script: npm run db:seed      # Insert baseline test data
14            env:
15              DATABASE_URL: $(TEST_DB_URL)
16
17  - stage: Tests
18    dependsOn: PrepareEnvironment
19    jobs:
20      - job: RunTests
21        steps:
22          - script: npx playwright test
23            env:
24              DATABASE_URL: $(TEST_DB_URL)
25              BASE_URL: $(STAGING_URL)

The seed script populates known, stable data. Tests that need dynamic data create it themselves.

Example seed script

TYPESCRIPT
1// scripts/db-seed.ts
2import { db } from '../src/database'
3
4async function seed() {
5  // Create base users (stable across runs)
6  await db.users.upsert({
7    where: { email: 'admin@test.com' },
8    create: {
9      id: '00000000-0000-0000-0000-000000000001',
10      email: 'admin@test.com',
11      role: 'admin',
12      name: 'Test Admin'
13    },
14    update: {}
15  })
16
17  // Create test products
18  await db.products.createMany({
19    data: SEED_PRODUCTS,
20    skipDuplicates: true
21  })
22
23  console.log('Seed complete')
24}
25
26seed().catch(console.error).finally(() => db.$disconnect())

---

Strategy 2: Factory pattern for dynamic data

Each test creates and destroys its own data:

TYPESCRIPT
1// tests/helpers/factory.ts
2import { randomUUID } from 'crypto'
3
4export class UserFactory {
5  private createdIds: string[] = []
6
7  async create(overrides = {}) {
8    const userData = {
9      id: randomUUID(),
10      email: `test-${randomUUID()}@example.com`,
11      name: 'Test User',
12      role: 'member',
13      ...overrides
14    }
15
16    const res = await fetch(`${process.env.API_URL}/admin/users`, {
17      method: 'POST',
18      headers: {
19        'Authorization': `Bearer ${process.env.ADMIN_TOKEN}`,
20        'Content-Type': 'application/json'
21      },
22      body: JSON.stringify(userData)
23    })
24
25    const user = await res.json()
26    this.createdIds.push(user.id)
27    return user
28  }
29
30  async cleanup() {
31    for (const id of this.createdIds) {
32      await fetch(`${process.env.API_URL}/admin/users/${id}`, {
33        method: 'DELETE',
34        headers: { 'Authorization': `Bearer ${process.env.ADMIN_TOKEN}` }
35      }).catch(err => console.warn(`Cleanup failed for ${id}: ${err.message}`))
36    }
37    this.createdIds = []
38  }
39}

TYPESCRIPT
1// In tests
2test.beforeEach(async () => {
3  userFactory = new UserFactory()
4})
5
6test.afterEach(async () => {
7  await userFactory.cleanup()
8})
9
10test('Admin can view user details', async ({ request }) => {
11  const user = await userFactory.create({ role: 'member' })
12  const res = await request.get(`/api/users/${user.id}`)
13  expect(res.status()).toBe(200)
14})

---

Storing test credentials securely

Never hardcode credentials in test code or YAML. Use Azure Key Vault with a service connection:

YAML
1steps:
2  - task: AzureKeyVault@2
3    displayName: Load test secrets
4    inputs:
5      azureSubscription: $(AZURE_SERVICE_CONNECTION)
6      KeyVaultName: 'myapp-test-keyvault'
7      SecretsFilter: 'TEST-DB-URL, TEST-ADMIN-TOKEN, STAGING-URL'
8      RunAsPreJob: true
9
10  - script: npx playwright test
11    env:
12      DATABASE_URL: $(TEST-DB-URL)          # Loaded from Key Vault
13      ADMIN_TOKEN: $(TEST-ADMIN-TOKEN)
14      BASE_URL: $(STAGING-URL)

---

Pre-run cleanup for resilience

Even with good cleanup, CI failures can leave orphaned test data. Add a pre-run cleanup:

TYPESCRIPT
1// scripts/cleanup-test-data.ts
2async function cleanupTestData() {
3  const cutoff = new Date(Date.now() - 24 * 60 * 60 * 1000) // 24 hours ago
4
5  // Delete users created by tests (identified by email pattern)
6  const deleted = await db.users.deleteMany({
7    where: {
8      email: { contains: '@example.com' },
9      createdAt: { lt: cutoff }
10    }
11  })
12  console.log(`Cleaned up ${deleted.count} stale test users`)
13}

Add this to your pipeline's preparation stage.

---

Common errors and fixes

Error: Database seed fails because tables don't exist yet Fix: Run db:migrate before db:seed. Migrations create the schema; seeds populate data. The order matters.

Error: Parallel test jobs conflict on shared test data Fix: Use the factory pattern with UUID-based identifiers. Each test creates its own records and doesn't depend on shared data.

Error: Test data cleanup fails silently and data accumulates Fix: Log cleanup results explicitly. Monitor the test database size over time. If it grows between pipeline runs, cleanup is failing. Use a pre-run cleanup as a safety net.

Error: Key Vault access fails in pipeline with "Forbidden" Fix: The service connection's service principal must have "Key Vault Secrets User" role on the Key Vault. Grant this in the Azure portal under Key Vault → Access control (IAM).