* Common autotuning test
* Autotuned model evaluation utilities
* Bugfixes and more enablement
* Working autotune profiling test
* Refactoring based on PR comments
Bugfixes to get tests passing
Refactor to remove tflite model for consistency
Black formatting
Linting and bugfixes
Add Apache license header
Use larger chunk size to read files
Explicitly specify LRU cache size for compatibility with Python 3.7
Pass platform to microTVM common tests
Better comment for runtime bound
Stop directory from being removed after session creation
* Use the actual Zephyr timing library
Use unsigned integer
Additional logging
Try negation
Try 64 bit timer
Use Zephyr's timing library
Fix linting
Enable timing utilities
* U3
Change-Id: Ibc088f19ad1dc9466fc368f8523baa30ee88b7d0
* addressed upstream comments
* Unit test added
Added unit test for InterfaceCNode::EmitConstantPool method
* [TIR.Constant] U1 usecase
Constants are now aggregated into one struct and initialized in default_lib0.c
file
Change-Id: I34d61f8139c8a92c06944fe990ba892a660476fd
Unit test fixed
Change-Id: I436e7b6d6b3064b3f8bbfbb048d4296b63a6b69c
* Refactored
Addressed:
* PoolInfo splitted to WorkspacePoolInfo and ConstantPoolInfo
* workspace_byte_alignment moved to ExecutorCodegenMetadata
* getModuleAlignment -> GetModuleAlignment
* GenerateInternalWorkspaceBuffers refactored
* reverted format change of src/tir/transforms/legalize_packed_calls.cc
* addressed comments for src/tir/usmp/analysis/extract_buffer_info.cc
* removed commented code from include/tvm/tir/usmp/utils.h
Change-Id: I7d1b32884b0e5992e2e00c7838c85e425d9c25fd
* more unit test fixes
Change-Id: I573a05fa1cb4037ae83691f7dff2c2724b1d7700
* More refactoring and unit test fixes
Added ConstantMemoryPools
Change-Id: If1e391c631575980564bca790ba33748c82d907f
* bugfix
Change-Id: Iacc7a9d734a505dfa0d8d32d23ea3f57e6de8582
* refactoring. added constant_alignment
added constant_alignment
unit tests updated
Change-Id: I378193cb9e675e352c61d96ff4e09655090053e1
* unit-test bugix
Change-Id: Ia4411d59c4a376c01326fed366cdb196a432899e
* unit test fix
Change-Id: Ia2077bdeb1d2c6c9827eeef90ab410ae31b8c4a4
* Added support for c++ runtime
* refactored
* renamed pools and consts
renamed pools and consts to workspace_pools and constant_pools
* addressed upstream comments
* addressed upstream comments-2
* addressed upstream comments-3
Some python tests are failing in the wheel. This PR skips them if the environment variable `WHEEL_TEST` is set.
This PR is related to https://github.com/tlc-pack/tlcpack/pull/115.
* [BYOC] Make CUTLASS BYOC integration 'Collage friendly'
(See https://discuss.tvm.apache.org/t/byoc-supporting-cutlass-byoc-with-collage/12796/6 for
context, which in turn is part of Collage (https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md).
Currently CUTLASS has four entry points:
- The usual 'partition_for_cutlass' partitioning function, using the
standard pattern table and pass machinery (see cutlass/build.py).
- A 'tune_cutlass_kernels' function which augments CUTLASS partition
functions with the results of building and running test kernels (see cutlass/build.py).
- A 'relay.ext.cutlass' external codegen function which inspects the
turning results and generates a CSourceModule for each partitions
(see cutlass/codegen.cc).
- A 'build_cutlass_kernels_vm' function which runs 'export_library' with
all the nvcc compiler options needed to build all the CSourceModules
(see cutlass/bild.py).
For Collage we'd like CUTLASS to have only two entry points: 'partition_for_cutlass',
and 'relay.ext.cutlass' or equivalent. This makes the CUTLASS external codegen integration
composable with other integrations, which in turn helps Collage avoid having to understand any
external codegen APIs other than the global pattern table and the custom compilation function/pass.
Collage also tends to end up requiring multiple partitions for the same backend since it is
more aggressive at mixing-and-matching smaller sub-graphs between backends. Thus we'd also like
to make sure all tuning, generated code and compilation overhead is shared between all such CUTLASS
partitions.
So, in this PR:
- We add all the CUTLASS-specific tuning and compilation options as new Target
attributes for the 'external codegen' "cutlass" TargetKind (cutlass/target.cc).
The user now has one place to provide those settings, and we've already done the
legwork to plumb the target instance.
- We replace 'relay.ext.cutlass' with a 'RelayToTIR' custom pass hook
'CompileForCutlass' (see cutlass/codegen.cc). This pass obviously can see all
the CUTLASS partitions in the IRModule, so we can now share tuning results
between them all and can be sure to generate a single CSourceModule. The pass can
also invoke the compiler to yield a StaticModule, which we've also already done the
legwork to support. In this way all CUTLASS-specific steps are handled at once.
- For convenience we supply 'finalize_modules' and 'finalize_modules_vm' which
invoke nvcc for final linking (using export_library as usual). However, there's now
nothing CUTLASS specific in those helpers other than their overriding of the 'compiler' to
be nvcc.
- test_cutlass.py is updated to use the new API.
Though this is a breaking change for existing users of the CUTLASS integration the
change is pretty minor, as shown in test_cutlass.py.
* - Masa's comments
* - Remove unnecessary save.
* [TIR, CUDA] Add pass to replace global to shared memory copy with cp.async
* add missing doc
* black
* missing src
* clang format
* clang format
* check against nested async scope
* [CI] Improved skip messages when using @tvm.testing.requires_*
Previously, the same message was given regardless of why a test
couldn't be run. This has been split up into separate checks for TVM
cmake options in `config.cmake`, enabled targets in `TVM_TEST_TARGETS`
environment variable, and checks for available hardware.
* Refactor to specify repeated feature marks, compile-only markers
* Fixed lint errors
* Import from contrib, not from a different import
* Removed use of requires_llvm() as a list of marks
* Corrected mark from requires_gpu to requires_cuda
* Adding missing "not"
* Added USE_CMSISNN as a requirement for corstone300.
* fixed hard-coded index in software pipeling
* fixed three-stage pipeline test
* add three stage pipelined gemm test
* refactor mma test
* use mma_4k schedule utility in test
* apply pipeling annotation
* black
* require ampere in test
Unlike ctypes, Cython by default do not release GIL when
calling into C API functions. This causes problems when the
function is long running. As the particular calling thread will
block other python threads by holding the GIL.
This PR explicitly releases GIL when calling into possible
long running functions. It fixes the timeout issue in
PopenPool which previously relied on another python thread
for timeout.
Added a regression test-case by changing sleep to sleep
in FFI, which previously will indefinitely block the popen tests.
* [Analysis] Exposed Analyzer::CanProveEqual to Python API
Checking for `analyizer.simplify(lhs-rhs) == 0` was a frequent pattern
in Python unit tests, and already had a utility function in the C++
public API. Exposing this utility function to Python allowed this
pattern to be cleaned up.
* Replaced more cases of .simplify with .can_prove_equal
* add get_c_struct_name() method to Metadata to distinguish struct type name in llvm
* add metadata serialization support to llvm codegen
* Organize MetadataQueuer into a separate file.
* Add DiscoverArraysVisitor to metadata_utils
* Fill DLTensor metadata in LegalizePackedCalls.
* Improve error message from Call asserts
* Pass non-String device_context down to codegen.
* this is necessary to allow CodeGenCPU to emit calls that include resource_handle.
* Scope usage of lvalue refs in LowerTVMBuiltin to avoid corrupt memory.
* test fixes
* Also fill preflattened_buffer_map (TODO, maybe don't do this)
* Fix C codegen.
* Set USMP elem_offset to 0.
* Clarify calculation of byte_offset from elem_offset.
* fix tests
* Fix arm compile warning
* Fix hexagon test.
* previously I believe we required interface_api == "c", but
this really means to generate C API bindings, and we are generating
"packed" bindings.
* I think "c" was chosen here because the distinction between
interface-api and use-unpacked-api is confusing. "c" interface-api
means to generate an entrypoint API for microcontrollers that
accepts bare data buffers. "packed" interface-api means to generate
a TVMBackendPackedCFunc entrypoint. use-unpacked-api forms the same
determination for the operator functions.
* A further confusion here is that there are two ways to call
"packed" operator functions: tir.tvm_builtin_call_packed and
tir.tvm_builtin_call_cpacked. This distinction describes whether or
not to late-bind calls via TVMBackendGetFuncFromEnv. Right now, AOT
only ever requires call_cpacked because target_host == target, and
for all suitable target_host, we expect a single DSO-exportable
runtime.Module. When we move away from this by introducing
heterogeneous target support to AOT, we can use this as a condition
to help us choose between call_cpacked and call_packed (and
possibly add a compile-time option to assert it is call_cpacked,
for situations where we really don't want call_packed).
* Document T.preflattened_buffer
* Fix test_aot_legalize_packed_calls
* Address manupa comments
* Fix convert_pool_allocations_to_offsets test.
* lint
* Fix T.preflattened_buffer
* Add preflattened_buffer_map to TIRTextPrinter
* Fix tests
* Fix BYOC
* Fix invoking C device API.
* remove comments
* Address Mousius comments
* lint
* lint
* Fix GMock linking on new CMake
* address masahi comment
Co-authored-by: Masahiro Masuda <masahi129@gmail.com>
* Make all required adjusts in the code to comply with the new version
* Upadte ci-lint to v0.71, based on tlcpackstaging/ci_lint:20220411-060305-45f3d4a52
* Register relay.ext.vitis_ai.available function
* Fix vitis-ai tests when running with USE_VITIS_AI OFF
* Replace skip_test with pytest skipif
* Add a function to see if vitis_ai is available
* Use requires_vitis_ai function for running tests
The Python and base image update for the i386 container changed the results of the various functions in `platform` as found in #10687. This updates them to work correctly with the new container and updates the relevant parts of the codebase to use the new check.
cc @masahi @mosius
Co-authored-by: driazati <driazati@users.noreply.github.com>
* [UnitTest] Disable ptx mma tests on unsupported nvcc versions.
- Modified `tvm.contrib.nvcc.get_cuda_version` to return a
`(major,minor,release)` tuple rather than a float.
- Implemented `tvm.testing.requries_nvcc_version` decorator to specify
the minimum `(major,minor,release)` version needed to run a unit
test.
- Applied decorated to unit tests in `test_tir_ptx_mma.py` that fail
on earlier nvcc versions.
* Fix lint errors.
* Updated a few of the cuda version checks.
* More lint fixes.
* Only compare major/minor in find_libdevice, not release version.
A follow-up from https://github.com/apache/tvm/pull/9188. The
`item.location` tuple contains `(filename, line_number, test_name)`,
where the `test_name` includes a string representation of all
parameters. This change preserves pytest's sorting of parametrized
values within a parametrized test, rather than sorting by strings.
* [Pytest] Sort unit tests before running.
By default, pytest will sort tests to maximize the re-use of fixtures.
However, this assumes that all fixtures have an equal cost to
generate, and no caches outside of those managed by pytest. A fixture
for a `tvm.testing.parameter` is effectively free, while a fixture
maintaining a cache of reference data
`tvm.testing.utils._fixture_cache` be quite large.
Since most of the TVM fixtures are specific to a python function, sort
the test ordering by python function, so that
tvm.testing.utils._fixture_cache can be cleared sooner rather than
later.
* Updated TestTargetAutoParametrization
When sorting the tests, the order of parametrizations may change.
Therefore, the tests checking for automatic target parametrization
shouldn't depend on order.
* [UnitTests][CMSISNN] Mark CMSISNN with skipif they are missing libraries
Show test as skipped, rather than failing test.
* Added tvm.testing.requires_cmsisnn
* [UnitTest] Added ids argument to tvm.testing.parameters
This matches the usage in `tvm.testing.parameter`, and allows for
parameter sets to be referred to by a single name.
* [Pytest] Fixed ordering issue of tvm.testing.parametrize_targets and known_failing_targets
If an explicit list of targets is given, then the
`known_failing_targets` decorator would fail to apply. This commit
resolves the issue, and cleans up all target-specific marks to apply
in `tvm.testing.plugin._add_target_specific_marks`.
* [UnitTest][Vulkan] Runnable relay unit tests on Vulkan
This commit allows the relay test suite to be run targeting Vulkan with
`TVM_TEST_TARGETS="vulkan -from_device=0" pytest tests/python/relay`. All
tests that require a specific environment are skipped if that environment
isn't present. All tests that are known to fail when running on Vulkan
are marked as expected failure, and will be tracked in
https://github.com/apache/tvm/issues/8903.
- Failures during code generation
- Type mismatches, boolean vs int8
- tests/python/relay/test_any.py::test_any_reduce
- tests/python/relay/test_op_level3.py::test_sparse_reshape
- tests/python/relay/test_op_level4.py::test_reduce_functions
- tests/python/relay/test_vm.py::test_cond
- tests/python/relay/test_vm.py::test_simple_if
- Incorrect strategy selection, picks NCHWc implemenation for NHWC layout
- tests/python/relay/test_op_level2.py::test_conv2d_run
- Unresolved CallNode operation
- tests/python/relay/test_op_level1.py::test_unary_op[erf/tan/atan]
- tests/python/relay/test_op_level3.py::test_scatter_add
- tests/python/relay/test_op_level3.py::test_segment_sum
- Generates 64-bit calls to GLSL that have only 16-/32-bit support
- tests/python/relay/test_op_grad_level1.py::test_log_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_softmax_grad
- tests/python/relay/test_op_grad_level1.py::test_unary_op
- tests/python/relay/test_op_grad_level10.py::test_cross_entropy_grad
- Codegen raises error for variable size
- tests/python/relay/test_any.py::test_any_batch_matmul
- tests/python/relay/test_any.py::test_any_conv2d_NCHWc
- tests/python/relay/test_any.py::test_any_dense
- Failures when running
- Numeric differences (observed on GTX 1650 with NVIDIA driver)
- tests/python/relay/test_op_level3.py::test_take
- tests/python/relay/test_op_level5.py::TestCropAndResize
- tests/python/relay/test_op_level5.py::TestResize1D
- tests/python/relay/test_op_level5.py::TestResize2D
- Adds the decorator `tvm.testing.requires_ethosn`
- Marks all tests in `tests/python/contrib/test_ethosn` as requiring
ethosn instead of directly checking `ethosn_available()`. This way,
they show up as skipped rather than passing.
- Marks test_compile_tflite_module_with_external_codegen as requiring
ethosn.
* [UnitTests][CoreML] Marked test_annotate as a known failure.
The unit tests in `test_coreml_codegen.py` haven't run in the CI
lately, so this test wasn't caught before. (See tracking issue
- Added `pytest.mark.xfail` mark to `test_annotate`.
- Added `tvm.testing.requires_package` decorator, which can mark tests
as requiring a specific python package to be available. Switched
from `pytest.importorskip('coremltools')` to
`requires_package('coremltools')` in `test_coreml_codegen.py` so
that all tests would explicitly show up as skipped in the report.
- Added `uses_gpu` tag to all tests in `test_coreml_codegen.py`, since
only ci_gpu has coremltools installed. In the future, if the ci_cpu
image has coremltools installed, this mark can be removed.
* [Pytest][TensorRT] Mark the TensorRT tests with tvm.testing.requires_cuda
Previously, the tests had an early bailout if tensorrt was disabled,
or if there was no cuda device present. However, the tests were not
marked with `pytest.mark.gpu` and so they didn't run during
`task_python_integration_gpuonly.sh`. This commit adds the
`requires_cuda` mark, and maintains the same behavior of testing the
tensorrt compilation steps if compilation is enabled, and running the
results if tensorrt is enabled.
In addition, some of the tests result in failures when run. These
have been marked with `pytest.mark.xfail`, and are being tracked in
issue #8901.
* [UnitTests] Expose TVM pytest helpers as plugin
Previously, pytest helper utilities such as automatic parametrization
of `target`/`dev`, or `tvm.testing.parameter` were only available for
tests within the `${TVM_HOME}/tests` directory. This PR extracts the
helper utilities into an importable plugin, which can be used in
external tests (e.g. one-off debugging).
* [UnitTests] Refactor the plugin-specific logic out into plugin.py.
* [UnitTests] Moved marker definition out to global variable.
* [UnitTests] Require cached fixtures to be copy-able, with opt-in.
Previously, any class that doesn't raise a TypeError in copy.deepcopy
could be used as a return value in a @tvm.testing.fixture. This has
the possibility of incorrectly copying classes inherit the default
object.__reduce__ implementation. Therefore, only classes that
explicitly implement copy functionality (e.g. __deepcopy__ or
__getstate__/__setstate__), or that are explicitly listed in
tvm.testing._fixture_cache are allowed to be cached.
* [UnitTests] Added TestCachedFixtureIsCopy
Verifies that tvm.testing.fixture caching returns copy of object, not
the original object.
* [UnitTests] Correct parametrization of cudnn target.
Previous checks for enabled runtimes were based only on the target
kind. CuDNN is the same target kind as "cuda", and therefore needs
special handling.
* Change test on uncacheable to check for explicit TypeError
Gavin Uberti