* complete celu op
* forgot to add test
* change order in convert_map, remove comment, delete import hiccup
Co-authored-by: CircleSpin <jocelyn@pop-os.localdomain>
* Fix use of fallback AutoTVM knobs
Previously knob values depended on order of explicit cfg update and cfg.define_split
calls in fallback mode
* Add test for define_split with fallback defined values
* alternative chunk op was implemented in pytorch frontend. aten::unsafe_chunk was added to op map in pytorch frontend
* chunk was replaced by new one in pytorch frontend. it is faster in 2.5 times
Co-authored-by: Valery Chernov <valery.chernov@deelvin.com>
* clean up typerel
* add layout transform when input is 3D
* add test
* update doc to clarify that only 2D input data is supported
* add weight_layout attribute in dense
* remove explicit layout transform from dense_alter_op.py
* Add DensePackInferCorrectLayout to insert layout transform
* relax type rel
* revert type rel relax and add check on dim
* introduce DensePackAttrs to avoid breaking dense op
* try fixing arm compute lib test
* Update tests/python/contrib/test_arm_compute_lib/test_dense.py
Co-authored-by: lhutton1 <35535092+lhutton1@users.noreply.github.com>
* formatting
Co-authored-by: lhutton1 <35535092+lhutton1@users.noreply.github.com>
* [Target][UnitTests] Look up target requirements based on tvm.target.Target
- Read target.kind.name instead of using string manipulation.
- Target device query on a non-existent target is no longer an error.
This occurs if expanding `vulkan -from_device=0` on a non-GPU
machine.
* [UnitTests] Added cuDNN target to default test targets
Some unit tests explicitly test cudnn in addition to
`tvm.testing.enabled_targets()`. This moved the cudnn checks into the
same framework as all other targets, and adds it to the default list
of targets to be run. Also, added `@tvm.testing.requires_cudnn` for
tests specific to cudnn.
* [UnitTests] pytest.xfail for CuDNN conv2d with asymmetric padding
* [Topi][CuDNN] Added handling of dilation to conv2d_cudnn
* [Topi] Skip dynamic batch matmul on cudnn, vulkan, opencl
Previously, cuda/nvptx targets were excluded. Changed it to look up
by target.kind.name, and to also exclude vulkan/opencl, as the dynamic
lookup currently doesn't work on those backends.
Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
* [Fix] Remove redundant cuda kernels caused by fusion of less_less_logical_or
* put the check function in reduction.py and add UT
* fix CI issue
* fix CI
* fix CI
Co-authored-by: saury <saury@saurydeMacBook-Pro.local>
Co-authored-by: saury <lifei59@meituan.com>
* Initial commit of API server impl.
* initial commit of api client
* Add TVM-side glue code to use Project API
* Change tvm.micro.Session to use Project API
* Rework how crt_config.h is used on the host.
* use template crt_config.h for host test runtime; delete
src/runtime/crt/host/crt_config.h so that it doesn't diverge from
the template
* bring template crt_config.h inline with the one actually in use
* rename to MAX_STRLEN_DLTYPE
* Create a dedicated TVM-side host crt_config.h in src/runtime/micro
* Modify Transport infrastructure to work with Project API
* Add host microTVM API server
* Zephyr implementation of microTVM API server
* move all zephyr projects to apps/microtvm/zephyr/template_project
* consolidate CcompilerAnnotator
* Allow model library format with c backend, add test.
* Update unit tests
* fix incorrect doc
* Delete old Zephyr build infrastructure
* Delete old build abstractions
* Delete old Transport implementations and simplify module
* lint
* ASF header
* address gromero comments
* final fixes?
* fix is_shutdown
* fix user-facing API
* fix TempDirectory / operator
* Update micro_tflite tutorial
* lint
* fix test_crt and test_link_params
* undo global micro import, hopefully fix fixture
* lint
* fix more tests
* Address tmoreau89 comments and mehrdadh comments
* fix random number generator prj.conf for physical hw
* uncomment proper aot option
* [Topi][Testing] Minor cleanup for python reference implementations
- Use input dtype for dilate/conv2d accumulate in python
impl. Previously, the python implementations of dilation and conv2d
would use numpy default dtype in some cases, rather than the input
data's dtype.
- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).
- Refactored to avoid duplication, use common get_pad_tuple functionality.
* [Topi][UnitTests] Added float16 tests to test_topi_dense.py
* [Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py
* [Topi][Float16] Added float16 tests for depthwise conv2d.
* [UnitTests] Explicitly set seed for float16 tests
Intended to avoid flaky test failures later due to rounding errors.
* [UnitTests] Fixed a few failing unit tests.
- ref_data must be a test fixture, not acquired through
request.getfixturevalue, in order to have the random_seed be known.
- dilate_python's return value didn't follow `out_dtype`.
- The test_topi_conv3d tests had the reference results computed in
float64, due to dilate_python() not respecting the input data type.
With the correct dtype, the tolerances needed to be slightly widened.
Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
* [Onnx][UnitTests] Excluded additional onnx tests
- The onnx tests `test_basic_convinteger`, `test_convinteger_with_padding`, `test_range_float_type_positive_delta_expanded`, and `test_range_int32_type_positive_delta_expanded` don't run correctly on CUDA targets, so they are added to the exclusion.
- Parametrized over the relative directory name, rather than the full directory name. This improves readability of the pytest output, and keeps the same parametrized test name across different python version.
- Changed the target-specific skips to check the target kind, rather than the full target string.
* [UnitTests] Apply correct requires_gpu() pytest marks for parametrized target
Prevoiusly, the addition of tvm.testing._target_to_requirement pytest marks
was handled by the parametrize_targets function. The
_auto_parametrize_target function assumed that a unit test that was already
parametrized had all markings needed. If a unit test was explicitly
parametrized using @pytest.mark.parametrize, these marks would be missing.
In most cases, this explicit use of @pytest.mark.parametrize('target', ...)
should be avoided, but has value in the case of marking with multiple
parameters with @pytest.mark.parametrize('target,other', ...). This use
case isn't yet supported by the tvm.testing.parameters function. Therefore,
if this occurs, detect it and add the appropriate marks.
* [UnitTest] Bugfix, applying requires_* markers to parametrized targets.
Initial implementation did work correctly with
@tvm.testing.parametrize_targets.
Also, went through all cases where "target" is used to parametrize on
something other than a target string, and renamed.
* [Onnx] Switched from using pytest.skip to tvm.testing.known_failing_targets
After merging of the `tvm.testing.parametrize_targets` and
`tvm.testing._auto_parametrize_target` code paths,
`known_failing_targets` can be used in both cases.
* [Testing] Enable `Target` object as argument to _target_to_requirement
Previously, tvm.testing._target_to_requirement required the argument
to be a string. This commit allows it to be either a string or a
`tvm.target.Target`.
* [Testing] Auto-target parametrization, handle pytest ParameterSet
If the unit test has already been parametrized with pytest.params to
add parameter-specific marks, respect those existing marks.
This can happen in some cases in the CI, uncertain yet what is causing
them. Maybe pytest-xdist related, but there's some difficulty in
reproducing it locally.
Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
* fuse dence sum
* remove excess copying
* dev LSTM in ONNX
* alternative implementation of LSTM in onnx frontend. It is quicker than current one without tuning
* LSTM_dev2 was implemented in onnx frontend
* LSTM dev in pytorch frontend
* LSTM cell implementation was transferred to common place. Unneccessary code was removed
* lint fixes
* Weights permutation for LSTM layer in onnx frontend
* LSTM cell description was added
* arguments and values were renamed. descriptions of some methods were added
* LSTM output shape and actvations input format were fixed in onnx frontend
* empty. tvm-ci test
* unbind method was transferred from onnx frontend to common.py
* unbind method was transferred from pytorch frontend to common.py
* lstm cell was transferred from op/layers.py to frontend/common.py
* clean up weight dictionary initialization
* fix pytorch frontend wrapper over unbind method
* minor fix of comments
* empty. tvm-ci test restart
* empty. tvm-ci test restart
Co-authored-by: Valery Chernov <valery.chernov@deelvin.com>
* [Runtime] Add graph_executor get_input_index API.
In graph_executor use case, user can use set_input with
input index to set input parameter, but there is no straight
forward way to get correct index number with input name, here
provide get_input_index API to do such work.
* Update python/tvm/contrib/graph_executor.py
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
* Update python/tvm/contrib/graph_executor.py
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
* Update src/runtime/graph_executor/graph_executor.cc
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
* Update python/tvm/contrib/graph_executor.py
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
* support scalars in quantize and requantize
* Add affine type support for ops with multipe output, use it in concat, move to header
* support new ops, refactor tests
* add more binary ops
fix pylint
fix black
black broke pylint
oops on black
* fix a typo in a branch and add a test that hits it
* improve comments
* Docker env for Arm® Ethos™-U55 Port
* Added Arm® Corstone™-300 Reference System for testing
* Added Arm® Ethos™-U driver stack
* Added installation of Arm® Vela.
Co-authored-by: Manupa Karunaratne <manupa.karunaratne@arm.com>
Change-Id: Ie3cc43943c876d95618a39887aa666da20bcb1e4
* Docker env for Arm® Ethos™-U55 Port
* Removes /opt/arm/cmake/bin from the path
* Parameterizes Arm® Ethos™-U55 driver stack version number
Change-Id: I2162b40f82241fd013643cbfa8847b60d7f4f5a1
* Docker env for Arm® Ethos™-U55 Port
* Adds ethosu as an extra to /python/gen_requirements.py
Change-Id: I2162b40f82241fd013643cbfa8847b60d7f4f5a1
* Docker env for Arm® Ethos™-U55 Port
* Added comment explaining why Vela version needs to be pinned to 2.1.1
Change-Id: I1ade280faa5274cca78899f4dae9e596b16fb5df
* Introduce --interface-api={c,packed} parameter
This introduces structures generated to provide a documented and stable user
friendly interface to a TVM generated model, as can be seen in the AOT
demo application:
```
struct tvmgen_default_inputs inputs = {
.input_1 = input_data,
};
struct tvmgen_default_outputs outputs = {
.output = output_data,
};
int ret_val = tvmgen_default_run(&inputs, &outputs, NULL, NULL);
```
To facilitate this, some other changes are included:
* Removed dependency on `aot_executor.{c,h}` in tests, pending the
discussion in the interface RFC as to whether we keep them.
* Moved creation of test DLTensor's into the AOT test utils, in future this
can be replaced by loading via the Python API or otherwise
* Introduce `parametrize_aot_options` which can be used to test
permutations of AOT which work together - for now this filters C
interface and packed operators
* Updated demo application to generate the header for demonstration
purposes, we should consider porting the demo application to Model
Library Format and using the toolchain in the Zephyr App via CMake
instead?
This patch builds upon the improvements @giuseros made to AOT testing
and name mangling from #8014
* Tweak metadata variable description and MLF target loop
* Remove direct usage of `relay::Var` in meta_data.h
This looks like the only place that could be causing the Windows CI failures, so trying removing the additional header in meta_data.h
* Linting fix
* Post-rebase files fixing
These tests were somehow transmuted in transit, I've updated them to the
most recent variant of the test helpers.
* Strip back interface API to just inputs and outputs
This removes any speculative structures from the generated code and cleans up some of the documentation.
* Add header guards and tweak documentation
* [TOPI][CUDA] Improve the performance of scatter_nd by:
1. Split into 2 kernels, one does the "Init" and another does the "Update".
Thus they can have different Grid/Block configurations to better utilize
SMs.
2. Use atomic_add instead of direct assignment, which could avoid the race
condtion when multiple indices point to the same location of the output
tensor. With this moidification, it's safe now to use more CUDA threads
to gain more parallelism.
* Fix python code format.
* FIX: [TOPI][CUDA] Improve the performance of scatter_nd #8479
- Split ScatterND kernel into 2 sub-kernels using ib.new_scope()
- Replace ib.for_range() with blockIdx.y
- Using atomic_add when mode == "add"
- Keep threadIdx.x less than max_threads of GPU
* Comment added
* Add fallback implementation when "mode=add" meets int64
- Atomic_add from CUDA doesn't support int64 data type
- Change "ind{i}" to "ind%d"%i, where names of relay.var could correctly display
* Python format
* Fix line too long
* CI pass
* Empty, for CI pass
* Empty, for CI pass
* Empty, for CI pass
* Empty, for CI pass
* Empty, for CI pass
* Exchange blockIdx.x and blockIdx.y
* check for Vulkan or metal
* Fallback to previous algorithm when mode==update
* Update python/tvm/topi/cuda/scatter.py
Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com>
* Assign TODO
* Swapping then and else block
Co-authored-by: wenxizhu <wenxizhu@tencent.com>
Co-authored-by: CaptainDuke <captainduke328@gmail.com>
Co-authored-by: Tristan Konolige <tristan.konolige@gmail.com>
* hotfix check_grad perf regression: lift compile out of hot loop
* hoist interpreter creation out of python closure, fix weird conv2d bug on arm cpu
* lint
* try one more fix
* Remove all attr::storage_scope usage
* pyformat
* fixed VTA tests
* Update TIR text printer to print storage_scope on allocate
* print storage scope in AllocateNode ReprPrinter
* Fixed accidently removed scope tag check
* remove unused function
Co-authored-by: masa <masa@pop-os.localdomain>
* Add basic support for batch matmul transpose
* Update
* Lint fix & add tf convert support
* Update
Lint fix
* Bug fix for qnn.batch_matmul
* Bug fix for tensorflow test
* Add grad support for batch_matmul
* Lint fix
Re-triggle CI
Bug fix
Re-triggle CI
Re-triggle CI
Re-triggle CI
* Fix issue in 'vectorize' function for 1D and 3D tensors
* Add pooling tests for channel last layouts
* Add support for more general layouts in "poolnd" implementation
* Reformat with 'black'
* Fix lint issues
Yuanjing Shi