Run Selenium Tests in Azure DevOps Pipelines

Running Selenium tests in Azure DevOps requires configuring headless browsers on the agent, setting up the correct WebDriver, and publishing test results in a format Azure DevOps understands. This guide covers complete working YAML for both Java (TestNG/JUnit) and Python (pytest) setups.

---

Prerequisites on the agent

Microsoft-hosted Ubuntu agents include:

• Chrome browser (regularly updated)
• ChromeDriver matching the Chrome version
• Java 17
• Python 3.10+
• Maven, Gradle

They do NOT include a display server, so all browser tests must run in headless mode.

---

Java + TestNG pipeline

Project structure

selenium-tests/
  pom.xml
  src/test/java/
    tests/
      LoginTest.java
      CheckoutTest.java
  testng.xml

pom.xml dependencies

XML
1<dependencies>
2  <dependency>
3    <groupId>org.seleniumhq.selenium</groupId>
4    <artifactId>selenium-java</artifactId>
5    <version>4.21.0</version>
6  </dependency>
7  <dependency>
8    <groupId>org.testng</groupId>
9    <artifactId>testng</artifactId>
10    <version>7.10.2</version>
11    <scope>test</scope>
12  </dependency>
13  <dependency>
14    <groupId>io.github.bonigarcia</groupId>
15    <artifactId>webdrivermanager</artifactId>
16    <version>5.8.0</version>
17    <scope>test</scope>
18  </dependency>
19</dependencies>

Headless Chrome setup in tests

JAVA
1// BaseTest.java
2import io.github.bonigarcia.wdm.WebDriverManager;
3import org.openqa.selenium.WebDriver;
4import org.openqa.selenium.chrome.ChromeDriver;
5import org.openqa.selenium.chrome.ChromeOptions;
6
7public class BaseTest {
8    protected WebDriver driver;
9
10    @BeforeMethod
11    public void setUp() {
12        WebDriverManager.chromedriver().setup();
13
14        ChromeOptions options = new ChromeOptions();
15        options.addArguments("--headless=new");   // Headless mode for CI
16        options.addArguments("--no-sandbox");       // Required in Docker/CI
17        options.addArguments("--disable-dev-shm-usage"); // Prevent memory issues
18        options.addArguments("--window-size=1920,1080");
19
20        driver = new ChromeDriver(options);
21    }
22
23    @AfterMethod
24    public void tearDown() {
25        if (driver != null) driver.quit();
26    }
27}

Azure Pipelines YAML — Java

YAML
1trigger:
2  branches:
3    include: [main]
4
5pool:
6  vmImage: ubuntu-latest
7
8stages:
9  - stage: SeleniumTests
10    displayName: Selenium Tests
11    jobs:
12      - job: TestNG
13        displayName: TestNG Selenium Suite
14        timeoutInMinutes: 30
15        steps:
16          - task: JavaToolInstaller@0
17            displayName: Setup Java 17
18            inputs:
19              versionSpec: '17'
20              jdkArchitectureOption: x64
21              jdkSourceOption: PreInstalled
22
23          - script: mvn --version && java --version
24            displayName: Verify Java setup
25
26          - script: |
27              mvn test \
28                -Dsurefire.rerunFailingTestsCount=1 \
29                -DBASE_URL=$(STAGING_URL)
30            displayName: Run Selenium tests
31            env:
32              STAGING_URL: $(STAGING_URL)
33
34          - task: PublishTestResults@2
35            displayName: Publish TestNG results
36            inputs:
37              testResultsFormat: JUnit
38              testResultsFiles: '**/surefire-reports/TEST-*.xml'
39              testRunTitle: Selenium TestNG — $(Build.BuildNumber)
40              mergeTestResults: true
41            condition: always()
42
43          - task: PublishPipelineArtifact@1
44            displayName: Upload screenshots on failure
45            inputs:
46              targetPath: test-screenshots
47              artifact: selenium-screenshots
48            condition: failed()

---

Python + pytest pipeline

Requirements

# requirements.txt
selenium==4.21.0
pytest==8.2.0
pytest-html==4.1.1
pytest-xdist==3.5.0   # For parallel execution

Headless Chrome in Python

PYTHON
1# conftest.py
2import pytest
3from selenium import webdriver
4from selenium.webdriver.chrome.options import Options
5from selenium.webdriver.chrome.service import Service
6
7@pytest.fixture(scope="function")
8def driver():
9    options = Options()
10    options.add_argument("--headless=new")
11    options.add_argument("--no-sandbox")
12    options.add_argument("--disable-dev-shm-usage")
13    options.add_argument("--window-size=1920,1080")
14
15    service = Service()  # WebDriverManager or system ChromeDriver
16    browser = webdriver.Chrome(service=service, options=options)
17    yield browser
18    browser.quit()

Azure Pipelines YAML — Python

YAML
1trigger:
2  branches:
3    include: [main]
4
5pool:
6  vmImage: ubuntu-latest
7
8stages:
9  - stage: SeleniumTests
10    jobs:
11      - job: Pytest
12        timeoutInMinutes: 30
13        steps:
14          - task: UsePythonVersion@0
15            inputs:
16              versionSpec: '3.11'
17
18          - script: pip install -r requirements.txt
19            displayName: Install dependencies
20
21          - script: |
22              pytest tests/ \
23                --junitxml=test-results/results.xml \
24                --html=test-results/report.html \
25                --self-contained-html \
26                -n auto \
27                -v
28            displayName: Run Selenium tests
29            env:
30              BASE_URL: $(STAGING_URL)
31
32          - task: PublishTestResults@2
33            displayName: Publish pytest results
34            inputs:
35              testResultsFormat: JUnit
36              testResultsFiles: test-results/results.xml
37              testRunTitle: Selenium pytest — $(Build.BuildNumber)
38            condition: always()
39
40          - task: PublishPipelineArtifact@1
41            inputs:
42              targetPath: test-results/report.html
43              artifact: pytest-html-report
44            condition: always()

---

Common errors and fixes

Error: DevToolsActivePort file doesn't exist in Chrome Fix: Add --no-sandbox and --disable-dev-shm-usage Chrome options. This error occurs in Linux environments without proper Chrome sandbox support.

Error: ChromeDriver only supports Chrome version 114 — version mismatch Fix: Use WebDriverManager (io.github.bonigarcia:webdrivermanager) in Java or webdriver-manager in Python — it automatically downloads the matching ChromeDriver version.

Error: Tests pass locally but fail in pipeline with "element not visible" Fix: Headless Chrome may render differently. Set explicit window size (--window-size=1920,1080) and add explicit waits (WebDriverWait) instead of Thread.sleep().

Error: Screenshots directory not found when trying to upload artifact Fix: Create the directory unconditionally: - script: mkdir -p test-screenshots before tests run. Upload artifacts with condition: always() not condition: failed().

Error: Maven build fails: "Tests run: 0, Failures: 0, Errors: 0, Skipped: 0" Fix: Check that test class names match the Surefire plugin pattern (default: Test*.java, *Test.java, *Tests.java). Add <includes><include>**/*Test.java</include></includes> to Surefire plugin config if using a different pattern.