Run Playwright Tests in Azure DevOps

Playwright integrates cleanly with Azure DevOps — it produces JUnit XML output natively, captures screenshots and traces on failure, and supports parallel execution across multiple agents. This guide covers a production-ready Playwright pipeline configuration.

---

Basic Playwright pipeline

YAML
1# azure-pipelines.yml
2trigger:
3  branches:
4    include: [main]
5
6pr:
7  branches:
8    include: [main]
9
10pool:
11  vmImage: ubuntu-latest
12
13steps:
14  - task: NodeTool@0
15    inputs:
16      versionSpec: '20.x'
17    displayName: Setup Node.js
18
19  - script: npm ci
20    displayName: Install npm packages
21
22  - script: npx playwright install --with-deps
23    displayName: Install Playwright browsers
24
25  - script: npx playwright test
26    displayName: Run Playwright tests
27    env:
28      BASE_URL: $(STAGING_URL)
29
30  - task: PublishTestResults@2
31    displayName: Publish results
32    inputs:
33      testResultsFormat: JUnit
34      testResultsFiles: playwright-results/results.xml
35      testRunTitle: Playwright — $(Build.BuildNumber)
36    condition: always()
37
38  - task: PublishPipelineArtifact@1
39    displayName: Upload HTML report
40    inputs:
41      targetPath: playwright-report
42      artifact: playwright-report
43    condition: always()

playwright.config.ts for CI

TYPESCRIPT
1import { defineConfig, devices } from '@playwright/test'
2
3export default defineConfig({
4  testDir: './tests',
5  fullyParallel: true,
6  forbidOnly: !!process.env.CI,      // Fail if test.only left in code
7  retries: process.env.CI ? 2 : 0,   // Retry twice in CI
8  workers: process.env.CI ? 4 : undefined,
9  reporter: [
10    ['junit', { outputFile: 'playwright-results/results.xml' }],
11    ['html', { outputFolder: 'playwright-report', open: 'never' }],
12    ['list'],
13  ],
14  use: {
15    baseURL: process.env.BASE_URL || 'http://localhost:3000',
16    trace: 'on-first-retry',     // Capture traces on retry
17    screenshot: 'only-on-failure',
18    video: 'on-first-retry',
19  },
20  projects: [
21    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
22    { name: 'firefox',  use: { ...devices['Desktop Firefox'] } },
23  ],
24})

---

Multi-stage pipeline with sharding

For large test suites, split tests across multiple agents using Playwright's built-in sharding:

YAML
1trigger:
2  branches:
3    include: [main]
4
5pool:
6  vmImage: ubuntu-latest
7
8stages:
9  - stage: Test
10    jobs:
11      # Run 4 shards in parallel
12      - job: Shard1
13        displayName: Tests shard 1/4
14        steps:
15          - template: templates/playwright-steps.yml
16            parameters:
17              shardIndex: 1
18              shardTotal: 4
19
20      - job: Shard2
21        displayName: Tests shard 2/4
22        steps:
23          - template: templates/playwright-steps.yml
24            parameters:
25              shardIndex: 2
26              shardTotal: 4
27
28      - job: Shard3
29        displayName: Tests shard 3/4
30        steps:
31          - template: templates/playwright-steps.yml
32            parameters:
33              shardIndex: 3
34              shardTotal: 4
35
36      - job: Shard4
37        displayName: Tests shard 4/4
38        steps:
39          - template: templates/playwright-steps.yml
40            parameters:
41              shardIndex: 4
42              shardTotal: 4
43
44      # Merge results from all shards
45      - job: MergeReports
46        displayName: Merge and publish reports
47        dependsOn: [Shard1, Shard2, Shard3, Shard4]
48        condition: always()
49        steps:
50          - task: NodeTool@0
51            inputs:
52              versionSpec: '20.x'
53          - script: npm ci
54          - task: DownloadPipelineArtifact@2
55            inputs:
56              artifact: shard-results
57              targetPath: all-results
58          - script: npx playwright merge-reports --reporter=html all-results
59            displayName: Merge HTML reports
60          - task: PublishPipelineArtifact@1
61            inputs:
62              targetPath: playwright-report
63              artifact: merged-playwright-report

Template templates/playwright-steps.yml:

YAML
1parameters:
2  - name: shardIndex
3    type: number
4  - name: shardTotal
5    type: number
6
7steps:
8  - task: NodeTool@0
9    inputs:
10      versionSpec: '20.x'
11  - script: npm ci
12  - script: npx playwright install --with-deps chromium
13  - script: |
14      npx playwright test \
15        --shard=${{ parameters.shardIndex }}/${{ parameters.shardTotal }} \
16        --reporter=blob
17    displayName: Playwright shard ${{ parameters.shardIndex }}/${{ parameters.shardTotal }}
18    env:
19      BASE_URL: $(STAGING_URL)
20  - task: PublishPipelineArtifact@1
21    inputs:
22      targetPath: blob-report
23      artifact: shard-results
24    condition: always()

---

Viewing test results and traces

After the pipeline runs:

1. Click the pipeline run → Tests tab
2. See all tests with pass/fail status, duration, error messages
3. Click a failed test to see the failure message
4. Download the playwright-report artifact → open index.html
5. In the HTML report, click a failed test → view screenshot, video, and trace

Playwright trace viewer:

Download the trace file (.zip) from the failed test, then run:

BASH
1npx playwright show-trace trace.zip

The trace viewer shows every action, network request, and DOM state — invaluable for diagnosing failures that don't reproduce locally.

---

Common errors and fixes

Error: Error: browserType.launch: Executable doesn't exist at /root/.cache/ms-playwright/... Fix: Add npx playwright install --with-deps chromium before the test step. The --with-deps flag installs OS-level dependencies (libgtk, etc.) needed on Ubuntu.

Error: Tests fail with "Target page, context or browser has been closed" Fix: Tests are timing out and Playwright force-closes the browser. Increase the timeout in playwright.config.ts or add explicit waits (await page.waitForLoadState('networkidle')).

Error: ENOENT: no such file or directory, open 'playwright-results/results.xml' Fix: Add mkdir -p playwright-results before running tests, or configure the output directory in playwright.config.ts to match what PublishTestResults expects.

Error: Playwright tests pass in the pipeline but HTML report is empty Fix: The reporter array in playwright.config.ts must include the HTML reporter. The open: 'never' option is important in CI — without it, Playwright tries to open a browser which fails on headless agents.

Error: Shard jobs all pass but MergeReports fails to find artifacts Fix: All shard jobs must upload their blob reports to the same artifact name. When using DownloadPipelineArtifact@2, ensure the artifact name matches exactly what the shard jobs upload.