Tianqi Chen
cfb98e938c
This PR simplifies Jenkins pytest execution around standard pytest-xdist behavior. - Runs each already-filtered CPU/GPU suite once with `-n auto`; the broad suite keeps load-group scheduling because its order-sensitive cases require it. - Removes external sharding, wrapper/profile code, JUnit XML generation and publication, the skipped-test XML consumer, obsolete suite naming, and orphaned helpers. - Retains one inert `task_clear_pytest.sh` entry point only because PR jobs evaluate their Jenkinsfile from the trusted base branch before checking out the PR; it performs no cleanup or reporting and can be removed after this pipeline lands. - Corrects stale broad-suite paths and explicit target guards, and migrates a scalar stride test to the current `T.handle` pointer semantics while preserving its negative lowering check. - Prevents nested MetaSchedule/XGBoost unit tests from multiplying CPU fanout without serializing the full suite. - Builds only the `tvm_runtime` target for the secondary GPU configuration and removes its unconsumed `gpu2` artifact upload. The result reduces parallelism to one layer managed by pytest-xdist while preserving GPU filtering and native failure visibility.
TVM Docker
This directory contains the TVM's docker infrastructure. We use docker to provide build environments for CI and images for demo. We need docker and nvidia-docker for GPU images.
Start Docker Bash Session
You can use the following helper script to start an interactive bash session with a given image_name.
/path/to/tvm/docker/bash.sh image_name
The script does the following things:
- Mount current directory to the same location in the docker container, and set it as home
- Switch user to be the same user that calls the bash.sh
- Use the host-side network
The helper bash script can be useful to build demo sessions.
Prebuilt Docker Images
You can use third party pre-built images for doing quick exploration with TVM installed.
For example, you can run the following command to launch tvmai/demo-cpu image.
/path/to/tvm/docker/bash.sh tvmai/demo-cpu
Then inside the docker container, you can type the following command to start the jupyter notebook
jupyter notebook
You can find some un-official prebuilt images in https://hub.docker.com/r/tlcpack/ . Note that these are convenience images and are not part of the ASF release.
Use Local Build Script
We also provide script to build docker images locally.
We use build.sh to build and (optionally) run commands
in the container. To build and run docker images, we can run the following
command at the root of the project.
./docker/build.sh image_name [command(optional)]
Here image_name corresponds to the docker defined in the
Dockerfile.image_name.
You can also start an interactive session by typing
./docker/build.sh image_name -it bash
The built docker images are prefixed by tvm., for example the command
./docker/build.sh image_name
produces the image tvm.ci_cpu that is displayed in the list of docker images
using the command docker images. To run an interactive terminal, execute:
./docker/bash.sh Dockerfile.ci_cpu
or
./docker/bash.sh ci_cpu echo hello tvm world
the same applies to the other images (./docker/Dockerfile.*).
The command ./docker/build.sh image_name COMMANDS is almost equivalent to
./docker/bash.sh image_name COMMANDS but in the case of bash.sh
a build attempt is not done.
The build command will map the tvm root to the corresponding location inside the container with the same user as the user invoking the docker command. Here are some common use examples to perform CI tasks.
-
build codes with CUDA support
./docker/build.sh ci_gpu bash -c "cd build && cmake -GNinja .. && ninja -j$(nproc)" -
do the python unittest
./docker/build.sh ci_gpu tests/scripts/task_python_unittest.sh -
build the documents. The results will be available at
docs/_build/html./docker/ci_build.sh ci_gpu bash -c "cd docs && make html"