* [UnitTest] Added ids argument to tvm.testing.parameters
This matches the usage in `tvm.testing.parameter`, and allows for
parameter sets to be referred to by a single name.
* [Pytest] Fixed ordering issue of tvm.testing.parametrize_targets and known_failing_targets
If an explicit list of targets is given, then the
`known_failing_targets` decorator would fail to apply. This commit
resolves the issue, and cleans up all target-specific marks to apply
in `tvm.testing.plugin._add_target_specific_marks`.
* [UnitTest][Vulkan] Runnable relay unit tests on Vulkan
This commit allows the relay test suite to be run targeting Vulkan with
`TVM_TEST_TARGETS="vulkan -from_device=0" pytest tests/python/relay`. All
tests that require a specific environment are skipped if that environment
isn't present. All tests that are known to fail when running on Vulkan
are marked as expected failure, and will be tracked in
https://github.com/apache/tvm/issues/8903.
- Failures during code generation
- Type mismatches, boolean vs int8
- tests/python/relay/test_any.py::test_any_reduce
- tests/python/relay/test_op_level3.py::test_sparse_reshape
- tests/python/relay/test_op_level4.py::test_reduce_functions
- tests/python/relay/test_vm.py::test_cond
- tests/python/relay/test_vm.py::test_simple_if
- Incorrect strategy selection, picks NCHWc implemenation for NHWC layout
- tests/python/relay/test_op_level2.py::test_conv2d_run
- Unresolved CallNode operation
- tests/python/relay/test_op_level1.py::test_unary_op[erf/tan/atan]
- tests/python/relay/test_op_level3.py::test_scatter_add
- tests/python/relay/test_op_level3.py::test_segment_sum
- Generates 64-bit calls to GLSL that have only 16-/32-bit support
- tests/python/relay/test_op_grad_level1.py::test_log_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_unary_op
- tests/python/relay/test_op_grad_level10.py::test_cross_entropy_grad
- Codegen raises error for variable size
- tests/python/relay/test_any.py::test_any_batch_matmul
- tests/python/relay/test_any.py::test_any_conv2d_NCHWc
- tests/python/relay/test_any.py::test_any_dense
- Failures when running
- Numeric differences (observed on GTX 1650 with NVIDIA driver)
- tests/python/relay/test_op_level3.py::test_take
- tests/python/relay/test_op_level5.py::TestCropAndResize
- tests/python/relay/test_op_level5.py::TestResize1D
- tests/python/relay/test_op_level5.py::TestResize2D
- Adds the decorator `tvm.testing.requires_ethosn`
- Marks all tests in `tests/python/contrib/test_ethosn` as requiring
ethosn instead of directly checking `ethosn_available()`. This way,
they show up as skipped rather than passing.
- Marks test_compile_tflite_module_with_external_codegen as requiring
ethosn.
* [UnitTests] Expose TVM pytest helpers as plugin
Previously, pytest helper utilities such as automatic parametrization
of `target`/`dev`, or `tvm.testing.parameter` were only available for
tests within the `${TVM_HOME}/tests` directory. This PR extracts the
helper utilities into an importable plugin, which can be used in
external tests (e.g. one-off debugging).
* [UnitTests] Refactor the plugin-specific logic out into plugin.py.
* [UnitTests] Moved marker definition out to global variable.
Lunderberg