Configuring CI

The action in Installation installs Python, the testkit, and warms the NVDA launcher cache. This page covers the decisions around it: which NVDA channel/version to test against, which Windows runner, and how this repository’s own CI configures both.

NVDA channel/version matrix

nvda-channel (or the plugin’s --nvda-channel / --nvda-version pytest flags) accepts:

Value Resolves to

stable

the current release, via nvaccess’s update-check endpoint

beta

the current beta, same endpoint

alpha

the newest alpha snapshot, scraped from nvaccess’s snapshot directory index — it publishes no checksum, so the download is unverified

a pinned version, e.g. 2026.1.1

that exact release build, also unverified

stable and beta downloads are checked against the SHA-1 nvaccess publishes; a mismatch deletes the file and raises rather than running an unverified executable. alpha and pinned versions have no published digest to check against — expect churn on alpha for that reason.

Test against more than one channel in CI so a new alpha or beta doesn’t surprise you at general availability. This repository’s own workflow runs both, and does not let alpha block the build:

e2e:
  runs-on: ${{ matrix.os }}
  strategy:
    fail-fast: false
    matrix:
      os: [windows-2025, windows-2022]
      channel: [stable, alpha]
  continue-on-error: ${{ matrix.channel == 'alpha' }}
  steps:
    - uses: actions/checkout@v5
    - uses: actions/setup-python@v6
      with:
        python-version: "3.13"
    - run: pip install -e ".[dev]"
    - name: Cache NVDA launchers
      uses: actions/cache@v4
      with:
        path: ~/.cache/nvda-testkit
        key: nvda-launcher-${{ matrix.os }}-${{ matrix.channel }}-${{ github.run_id }}
        restore-keys: nvda-launcher-${{ matrix.os }}-${{ matrix.channel }}-
    - run: nvda-testkit doctor
    - run: pytest tests_e2e/ -v --nvda-channel=${{ matrix.channel }}

fail-fast: false matters as much as continue-on-error here: without it, a failure on one cell of the matrix cancels the others before they report anything.

Windows runner setup

A real NVDA needs a real Windows desktop session — there is no headless mode. Pick the runner image that matches what your users run:

Runner Use it for

windows-2025

current Windows Server baseline

windows-2022

Windows 10 22H2, including builds still on Extended Security Updates

This repository’s own e2e suite runs both, per the matrix above. If your add-on has no ESU users to support, windows-2025 alone is enough.

Minimal job:

jobs:
  e2e:
    runs-on: windows-2025
    steps:
      - uses: actions/checkout@v5
      - uses: zirekhq/nvda-addon-testkit@v1
        with:
          nvda-channel: stable
      - run: scons dist
      - run: pytest tests_e2e/ -v

Only one NVDA per job

tests_e2e/ tests run serially: only one NVDA can own a desktop session, so the plugin refuses to start under pytest-xdist with more than one worker (pytest -n 2 tests_e2e/ fails fast with a UsageError rather than producing nonsense). Shard across CI jobs — the matrix above, one job per (os, channel) pair — instead of workers within one job.

Diagnose the runner before trusting a failure

nvda-testkit doctor checks the current machine without starting a full NVDA session: platform, whether the spy add-on bundle was built, the launcher cache, and whether every channel currently resolves. Run it as a step before your test step — a red doctor output means the environment is broken, not your add-on:

- run: nvda-testkit doctor
- run: pytest tests_e2e/ -v --nvda-channel=stable

Capture artifacts on failure

Point nvda-testkit’s `out-dir (default testOutput/) at an uploaded artifact so a failing run leaves behind the NVDA log, not just a pytest traceback:

- run: pytest tests_e2e/ -v
- if: failure()
  uses: actions/upload-artifact@v5
  with:
    name: e2e-artifacts
    path: testOutput/
    if-no-files-found: warn

See Troubleshooting for what to look for once you have that log.

Want to help? Learn how to contribute to the ZirekHQ docs ›