Files
Shushi Hong 0cc110ecd2 [CI] Bump CI at the Ubuntu 24.04 images and re-enable USE_Z3 (#19911)
This pr switches CI to the Ubuntu 24.04 (noble) images. Bump ci_tag in
ci/jenkins/docker-images.ini to 20260629-192919-24bbfd2e -- the images
built from #19893 (ci_cpu/ci_arm/ci_wasm/ci_gpu on Ubuntu 24.04, whose
default g++ is gcc-13, giving full C++20 support).

Also, this pr re-enables Z3 (AUTO). #19828 temporarily set USE_Z3=OFF
(in CMakeLists.txt and the pyproject wheel build) to dodge a z3-static
build failure. The CI image now ships z3-static (#19835), so this
restores USE_Z3=AUTO: the Z3-backed Analyzer proving is enabled when
z3-static is available and silently skipped otherwise.

While Z3 stayed disabled, PrimExprNode::ty became a method, leaving two
stale field accesses in z3_prover.cc's IsZ3SupportedExpr (only compiled
under TVM_USE_Z3). Fixed expr->ty -> expr->ty().

Verification:
- The Ubuntu 24.04 images (#19893) built successfully for ci_cpu/ci_arm/
ci_wasm/ci_gpu (the GPU image includes ROCm 6.4.4 and the CUDA 24.04
base).
- Re-enabling Z3 was validated with a build-only wheel run: all four
wheels (Linux x86_64/aarch64 manylinux_2_28, macOS arm64, Windows) build
green with z3-static compiled and linked, confirming the earlier
z3-static link concern is resolved on the current toolchain.
2026-07-14 17:27:00 -04:00
..

Apache TVM Continuous Integration (CI)

Overview

TVM's Continuous Integration is responsible for verifying the code in apache/tvm and testing PRs before they merge to inform TVM contributors and committers. These jobs are essential to keeping the TVM project in a healthy state and preventing breakages. CI in TVM is broken into these pieces:

  • Lint scripts in tests/lint.
  • The tests themselves, all of which live underneath tests.
  • Definitions of test suites, with each suite defined as a separate task_ script in tests/scripts.
  • Scripts and automation ci/scripts.
  • The linux test sequence (in Jenkinsfile), which lints and builds TVM and runs test suites using Docker on Linux.
  • The Windows and Mac test sequences (in .github/actions).
  • GitHub Actions that support the code review process (in .github/actions).
  • Tools to reproduce the CI locally (in tests/scripts).
  • Infrastructure-as-Code that configures the cloud services that provide Jenkins for the TVM CI (in the tlc-pack/ci repo).

CI Documentation Index

The CI documentation belongs with the implementation it describes. To make that concrete, the documentation is split like so:

  1. An overview of the CI is in this file.
  2. User-facing documentation lives in apache/tvm's docs/contribute sub-directory and is served on the TVM docs site.
  3. Documentation of the tools that run TVM's various regression tests locally and the test suites are in this sub-directory.
  4. Documentation of the cloud services and their configuration lives in the tlc-pack/ci repo.

Jenkins

Jenkins runs all of the Linux-based TVM CI-enabled regression tests. This includes tests against accelerated hardware such as GPUs. It excludes those regression tests that run against hardware not available in the cloud (those tests aren't currently exercised in TVM CI). The tests run by Jenkins represent most of the merge-blocking tests (and passing Jenkins should mostly correlate with passing the remaining Windows/Mac builds).

GitHub Actions

GitHub Actions is used to run Windows jobs, MacOS jobs, and various on-GitHub automations. These are defined in .github/workflows. These automations include bots to:

https://github.com/apache/tvm/actions has the logs for each of these workflows. Note that when debugging these workflows changes from PRs from forked repositories won't be reflected in the PR. These should be tested in the forked repository first and linked in the PR body.

Docker Images

Each CI job runs most of its work inside a Docker container, built from files in the docker/ folder. These files are built nightly in Jenkins via the tvm-docker job. The images for these containers are hosted in the tlcpack Docker Hub and referenced in the jenkins/templates. These can be inspected and run locally via standard Docker commands.

ci-docker-staging

The ci-docker-staging branch is used to test updates to Docker images and Jenkinsfile changes. When running a build for a normal PR from a forked repository, Jenkins uses the code from the PR except for the Jenkinsfile itself, which comes from the base branch. When branches are built, the Jenkinsfile in the branch is used, so a committer with write access must push PRs to a branch in apache/tvm to properly test Jenkinsfile changes. If your PR makes changes to the Jenkinsfile, make sure to @ a committer and ask them to push your PR as a branch to test the changes.

Jenkins CI

TVM uses Jenkins for running Linux continuous integration (CI) tests on branches and pull requests through a build configuration specified in a Jenkinsfile. Other jobs run in GitHub Actions for Windows and MacOS jobs.

Jenkinsfile

The template files in this directory are used to generate the Jenkinsfile used by Jenkins to run CI jobs for each commit to PRs and branches.

To regenerate the Jenkinsfile, run make in the ci/jenkins dir.