* [Analysis] Exposed Analyzer::CanProveEqual to Python API
Checking for `analyizer.simplify(lhs-rhs) == 0` was a frequent pattern
in Python unit tests, and already had a utility function in the C++
public API. Exposing this utility function to Python allowed this
pattern to be cleaned up.
* Replaced more cases of .simplify with .can_prove_equal
* Make all required adjusts in the code to comply with the new version
* Upadte ci-lint to v0.71, based on tlcpackstaging/ci_lint:20220411-060305-45f3d4a52
* Register relay.ext.vitis_ai.available function
* Fix vitis-ai tests when running with USE_VITIS_AI OFF
* Replace skip_test with pytest skipif
* Add a function to see if vitis_ai is available
* Use requires_vitis_ai function for running tests
The Python and base image update for the i386 container changed the results of the various functions in `platform` as found in #10687. This updates them to work correctly with the new container and updates the relevant parts of the codebase to use the new check.
cc @masahi @mosius
Co-authored-by: driazati <driazati@users.noreply.github.com>
* [UnitTest] Disable ptx mma tests on unsupported nvcc versions.
- Modified `tvm.contrib.nvcc.get_cuda_version` to return a
`(major,minor,release)` tuple rather than a float.
- Implemented `tvm.testing.requries_nvcc_version` decorator to specify
the minimum `(major,minor,release)` version needed to run a unit
test.
- Applied decorated to unit tests in `test_tir_ptx_mma.py` that fail
on earlier nvcc versions.
* Fix lint errors.
* Updated a few of the cuda version checks.
* More lint fixes.
* Only compare major/minor in find_libdevice, not release version.
* [UnitTests][CMSISNN] Mark CMSISNN with skipif they are missing libraries
Show test as skipped, rather than failing test.
* Added tvm.testing.requires_cmsisnn
* [UnitTest] Added ids argument to tvm.testing.parameters
This matches the usage in `tvm.testing.parameter`, and allows for
parameter sets to be referred to by a single name.
* [Pytest] Fixed ordering issue of tvm.testing.parametrize_targets and known_failing_targets
If an explicit list of targets is given, then the
`known_failing_targets` decorator would fail to apply. This commit
resolves the issue, and cleans up all target-specific marks to apply
in `tvm.testing.plugin._add_target_specific_marks`.
* [UnitTest][Vulkan] Runnable relay unit tests on Vulkan
This commit allows the relay test suite to be run targeting Vulkan with
`TVM_TEST_TARGETS="vulkan -from_device=0" pytest tests/python/relay`. All
tests that require a specific environment are skipped if that environment
isn't present. All tests that are known to fail when running on Vulkan
are marked as expected failure, and will be tracked in
https://github.com/apache/tvm/issues/8903.
- Failures during code generation
- Type mismatches, boolean vs int8
- tests/python/relay/test_any.py::test_any_reduce
- tests/python/relay/test_op_level3.py::test_sparse_reshape
- tests/python/relay/test_op_level4.py::test_reduce_functions
- tests/python/relay/test_vm.py::test_cond
- tests/python/relay/test_vm.py::test_simple_if
- Incorrect strategy selection, picks NCHWc implemenation for NHWC layout
- tests/python/relay/test_op_level2.py::test_conv2d_run
- Unresolved CallNode operation
- tests/python/relay/test_op_level1.py::test_unary_op[erf/tan/atan]
- tests/python/relay/test_op_level3.py::test_scatter_add
- tests/python/relay/test_op_level3.py::test_segment_sum
- Generates 64-bit calls to GLSL that have only 16-/32-bit support
- tests/python/relay/test_op_grad_level1.py::test_log_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_unary_op
- tests/python/relay/test_op_grad_level10.py::test_cross_entropy_grad
- Codegen raises error for variable size
- tests/python/relay/test_any.py::test_any_batch_matmul
- tests/python/relay/test_any.py::test_any_conv2d_NCHWc
- tests/python/relay/test_any.py::test_any_dense
- Failures when running
- Numeric differences (observed on GTX 1650 with NVIDIA driver)
- tests/python/relay/test_op_level3.py::test_take
- tests/python/relay/test_op_level5.py::TestCropAndResize
- tests/python/relay/test_op_level5.py::TestResize1D
- tests/python/relay/test_op_level5.py::TestResize2D
- Adds the decorator `tvm.testing.requires_ethosn`
- Marks all tests in `tests/python/contrib/test_ethosn` as requiring
ethosn instead of directly checking `ethosn_available()`. This way,
they show up as skipped rather than passing.
- Marks test_compile_tflite_module_with_external_codegen as requiring
ethosn.
* [UnitTests][CoreML] Marked test_annotate as a known failure.
The unit tests in `test_coreml_codegen.py` haven't run in the CI
lately, so this test wasn't caught before. (See tracking issue
- Added `pytest.mark.xfail` mark to `test_annotate`.
- Added `tvm.testing.requires_package` decorator, which can mark tests
as requiring a specific python package to be available. Switched
from `pytest.importorskip('coremltools')` to
`requires_package('coremltools')` in `test_coreml_codegen.py` so
that all tests would explicitly show up as skipped in the report.
- Added `uses_gpu` tag to all tests in `test_coreml_codegen.py`, since
only ci_gpu has coremltools installed. In the future, if the ci_cpu
image has coremltools installed, this mark can be removed.
* [Pytest][TensorRT] Mark the TensorRT tests with tvm.testing.requires_cuda
Previously, the tests had an early bailout if tensorrt was disabled,
or if there was no cuda device present. However, the tests were not
marked with `pytest.mark.gpu` and so they didn't run during
`task_python_integration_gpuonly.sh`. This commit adds the
`requires_cuda` mark, and maintains the same behavior of testing the
tensorrt compilation steps if compilation is enabled, and running the
results if tensorrt is enabled.
In addition, some of the tests result in failures when run. These
have been marked with `pytest.mark.xfail`, and are being tracked in
issue #8901.
* [UnitTests] Expose TVM pytest helpers as plugin
Previously, pytest helper utilities such as automatic parametrization
of `target`/`dev`, or `tvm.testing.parameter` were only available for
tests within the `${TVM_HOME}/tests` directory. This PR extracts the
helper utilities into an importable plugin, which can be used in
external tests (e.g. one-off debugging).
* [UnitTests] Refactor the plugin-specific logic out into plugin.py.
* [UnitTests] Moved marker definition out to global variable.
* [UnitTests] Require cached fixtures to be copy-able, with opt-in.
Previously, any class that doesn't raise a TypeError in copy.deepcopy
could be used as a return value in a @tvm.testing.fixture. This has
the possibility of incorrectly copying classes inherit the default
object.__reduce__ implementation. Therefore, only classes that
explicitly implement copy functionality (e.g. __deepcopy__ or
__getstate__/__setstate__), or that are explicitly listed in
tvm.testing._fixture_cache are allowed to be cached.
* [UnitTests] Added TestCachedFixtureIsCopy
Verifies that tvm.testing.fixture caching returns copy of object, not
the original object.
* [UnitTests] Correct parametrization of cudnn target.
Previous checks for enabled runtimes were based only on the target
kind. CuDNN is the same target kind as "cuda", and therefore needs
special handling.
* Change test on uncacheable to check for explicit TypeError
Florin Blanaru