DevOps#test-automation-framework #azure-devops #framework-design #ci-cd #page-object-model #test-architecture

Test Automation Framework Design for Azure DevOps Pipelines

How to design a test automation framework specifically optimised for Azure DevOps pipelines. Covers layer separation, configuration management, environment abstraction, reporting, and CI/CD-specific patterns.

March 20, 2026InnovateBits4 min read

ShareX / Twitter LinkedIn

A framework that works well locally often breaks in CI for reasons that have nothing to do with the tests themselves: hardcoded paths, environment assumptions, sequential test dependencies. Designing for Azure DevOps from the start avoids those pitfalls.

Framework layers

┌─────────────────────────────────────────┐
│         Test Cases (specs)              │  Business scenarios
├─────────────────────────────────────────┤
│         Page Object Model               │  UI abstractions
├─────────────────────────────────────────┤
│         Test Helpers & Fixtures         │  Data, auth, utilities
├─────────────────────────────────────────┤
│         Configuration Layer            │  Env vars, settings
├─────────────────────────────────────────┤
│         Driver / Client Layer           │  Browser, HTTP client
└─────────────────────────────────────────┘

Each layer has a single responsibility. Tests don't directly manipulate the browser. Pages don't know about test data. Config doesn't contain business logic.

Configuration management for multiple environments

// config/config.ts
type Environment = 'local' | 'staging' | 'uat' | 'production'
 
interface EnvConfig {
  baseUrl: string
  apiBaseUrl: string
  adminEmail: string
  adminPassword: string
  defaultTimeout: number
}
 
const configs: Record<Environment, EnvConfig> = {
  local: {
    baseUrl: 'http://localhost:3000',
    apiBaseUrl: 'http://localhost:4000',
    adminEmail: 'admin@local.test',
    adminPassword: 'localpass',
    defaultTimeout: 10_000,
  },
  staging: {
    baseUrl: process.env.STAGING_URL || '',
    apiBaseUrl: process.env.STAGING_API_URL || '',
    adminEmail: process.env.STAGING_ADMIN_EMAIL || '',
    adminPassword: process.env.STAGING_ADMIN_PASSWORD || '',
    defaultTimeout: 20_000,
  },
  uat: {
    baseUrl: process.env.UAT_URL || '',
    apiBaseUrl: process.env.UAT_API_URL || '',
    adminEmail: process.env.UAT_ADMIN_EMAIL || '',
    adminPassword: process.env.UAT_ADMIN_PASSWORD || '',
    defaultTimeout: 25_000,
  },
  production: {
    baseUrl: 'https://app.com',
    apiBaseUrl: 'https://api.app.com',
    adminEmail: '',
    adminPassword: '',
    defaultTimeout: 30_000,
  },
}
 
const env = (process.env.TEST_ENV as Environment) || 'local'
export const config = configs[env]

Page Object Model

// pages/LoginPage.ts
import { Page, Locator, expect } from '@playwright/test'
 
export class LoginPage {
  readonly page: Page
  readonly emailInput: Locator
  readonly passwordInput: Locator
  readonly signInButton: Locator
  readonly errorMessage: Locator
 
  constructor(page: Page) {
    this.page = page
    this.emailInput    = page.getByTestId('email-input')
    this.passwordInput = page.getByTestId('password-input')
    this.signInButton  = page.getByRole('button', { name: 'Sign in' })
    this.errorMessage  = page.getByTestId('error-message')
  }
 
  async navigate() {
    await this.page.goto('/login')
    await expect(this.emailInput).toBeVisible()
  }
 
  async login(email: string, password: string) {
    await this.emailInput.fill(email)
    await this.passwordInput.fill(password)
    await this.signInButton.click()
  }
 
  async expectError(message: string) {
    await expect(this.errorMessage).toContainText(message)
  }
}

Test data factory

// factories/user-factory.ts
import { randomUUID } from 'crypto'
import { config } from '../config/config'
 
let adminToken: string | null = null
 
async function getAdminToken(): Promise<string> {
  if (adminToken) return adminToken
  const res = await fetch(`${config.apiBaseUrl}/auth/login`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ email: config.adminEmail, password: config.adminPassword }),
  })
  const { token } = await res.json()
  adminToken = token
  return token
}
 
export async function createTestUser(overrides: Partial<User> = {}): Promise<User> {
  const token = await getAdminToken()
  const userData = {
    id: randomUUID(),
    email: `test-${randomUUID()}@example.com`,
    name: 'Test User',
    role: 'member',
    ...overrides,
  }
  const res = await fetch(`${config.apiBaseUrl}/users`, {
    method: 'POST',
    headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' },
    body: JSON.stringify(userData),
  })
  return res.json()
}
 
export async function deleteTestUser(userId: string): Promise<void> {
  const token = await getAdminToken()
  await fetch(`${config.apiBaseUrl}/users/${userId}`, {
    method: 'DELETE',
    headers: { Authorization: `Bearer ${token}` },
  })
}

Pipeline integration patterns

Environment-specific variable groups

Library
  ├── staging-env        (staging URLs, non-secret config)
  ├── staging-secrets    (credentials — secret variables)
  ├── uat-env
  └── uat-secrets

variables:
  - group: staging-env
  - group: staging-secrets
  - name: TEST_ENV
    value: staging

Fail fast, then report

steps:
  - script: npx jest tests/smoke  # Fast health check (30 sec)
    displayName: Smoke (fail fast)
 
  - script: npx jest tests/       # Full suite
    displayName: Full regression
    condition: succeeded()
 
  - task: PublishTestResults@2    # Always publish
    condition: always()

Common errors and fixes

Error: Config values are undefined in CI but work locally Fix: Check the variable group is linked to the pipeline AND the pipeline is authorised to use it. In Library → Variable Group → Pipeline permissions.

Error: Factory creates data but cleanup fails silently Fix: Wrap cleanup in try/catch and log failures. If cleanup consistently fails, add a pre-run cleanup step that deletes all test-prefixed records older than 24 hours.

Error: POM locators break after UI update Fix: Prefer data-testid attributes over CSS selectors or XPath. Ask the development team to add data-testid attributes to key elements. These are stable across UI redesigns.

Free newsletter

Stay ahead in AI-driven QA

Get practical tutorials on test automation, AI testing, and quality engineering — straight to your inbox. No spam, unsubscribe any time.