The current meta-schedule uses a PostProc `RewriteUnboundBlock` to auto-bind blocks to threads. However, it's a post proc, which means there are no search opportunities, and always splits with `factor=1024`.
This PR adds a new search rule called `AutoBind` to do a similar thing to bind threads with sampled factors. Also with a corresponding mutator.
After applying this rule, we get some positive perf results (on RTX-3080):
Element-wise: from 2.76 us to 2.48 us
Conv2d Winograd: from 29.45 us to 18.96 us (ansor 22.00 us)
Resnet18: from 0.591 ms to 0.531 ms (ansor 0.565 ms)
* [Schedule] Allowed typing.Tuple in tir.schedule._type_checker
Previously, `typing.Tuple` annotations could not be used with
`tir.schedule._type_checker.type_checked` annotations. This allows
`Tuple` type annotations to be type-checked.
* Revert change, allow tuples input as List arguments
* Suppress mypy errors
Directly interacting with a type object would otherwise cause some
false positives.
* Corrected unit test for allowing tuples to be used as typing.List
* Represent multi-type lists as List[Union[...]] instead of List[Any]
This gives a better error message and plays nicely with _type2str,
since `typing.Any` doesn't have a `__name__` field.
* Fix function number datatype from char to uint16_t
rewrite the modified part to pass lint check
Use 2 bytes for func num in fun_registry
Fix errors in linter
Add the declaration of the helper functions
set 2 bytes for func num in func_registry test units
pass num_func by value
This commit change the datatype of the number of the function from 1 Byte to 2 Bytes.
Besides, I use some helper functions to access the number of function and the first function name.
* Fix aot_executor_module to unbreak CI.
* Fix GraphExecutorModule.
* Remove graph_json_to_c_func_registry.
* No longer needed and not called anywhere.
* Superseded by emitting the FuncRegistry directly in codegen.
Co-authored-by: 嚴中璟 <a1245967@gmail.com>
* [microNPU] Fix bug in channels extraction in the matcher
If the input tensor layout is in NHCWB16, we were passing W value
instead of the channels to get_valid_block_configs.
* Add test for conv2d
* [TFLite] Add support to int16 data type in TFLite frontend
Add support for int16 data type and int64 biases/accumulators in
the TFLite frontend.
Adjusts TFLite tests to cover int16 convolutions and element-wise;
Fixes a minor typo negtive->negative in the element-wise tests.
* Update src/relay/qnn/op/convolution.cc
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
* [TVMC][ETHOSN] Improve target string to avoid duplication
Improves the TVMC target string to avoid duplication of the
NPU variant. The new target string will require the just the NPU
name followed by -variant=n78. The old target string is deprecated
and will be removed in a subsequent version of TVM.
Change-Id: I4638f36788df3f478435ac13d3531aad2b23f204
* fix linting
Change-Id: I76a9da511899f24a163be669877605cd1a440022
* fix make variant functions and update test error message
Change-Id: Iff553d4b255c0ce0b86bad42eaa94ee9b1c62508
* [TVMScript] Support function call to help construct AST
* add test
* update test
* more comment
* fix for avoiding Buffer.vload(...) case
* update parse error msg
* wrap func call with try / catch, emit error msg
* silence pylint
* support ConvTranspose explicitly specified output_shape
* fix unit test case
* fix lint test
* retest
* fix code error
* fix lint test
* update test
* retest
* fix test onnx official tests
This PR allowed users to set logging level without giving a logger config. Previous implementation hard-coded `logging.INFO` as the default logging level and requires a logger config to change it. Now the logging level and handlers can be inherited from the current `tvm.meta_schedule` logger setting.
* add relay.f.frontend.fm_oneflow support cnns
* support cuda
* fix mobilenetv2 and reviews
* fix: model without meta info
* support eager and yolo, add test
* fix: license
* add: tutorials
* fix: support new graph
* fix some comments
* refine
* fix concat op convert bug
* refine
* refine
* change cuda to cpu
* fix bug
* fix ci error in tvm
* fix pylint check
* delete useless file
* add skimage package in docker
* fix ci error
* fix bug
* add oneflow fronted test in ci
* merge conflict
* fix tutorial
* try to find error in ci
* revert
* merge conflict
* black oneflow
* Delete from_oneflow.py
* restruct oneflow fronted
* support vision-transformer
* black format
* update black version and reformat
* fix ci error
* fix doc error
* fix gpu fronted test failed
Co-authored-by: hhhfccz <hjk1938927583@163.com>
* [ROOFLINE] Roofline analysis over RPC
Run roofline analysis on remote devices if requested. Peak flops and
peak bandwidth estimation are done on the remote device.
* allocate testing arrays directly on device and randomly fill
* forgot to include remote
* lower flops ratio, machine may be using multiple threads
* forgot fill
* [QNN] Enable constant folding for QNN operations.
This commit enables constant folding for QNN operations.
This functionalty is disabled by default, use fold_qnn=True to enable.
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [NFC] Fixed comments
* Added more unit tests for QNN opers in constant folding pass.
* Address PR feedbacks
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [Debug] Error logging in DetectIterMap
* [Affine] Allowed PrimExpr argument to NormalizeIterMapToExpr
This allows it to be used for any expression containing an
`IterMapExpr`, not just expressions whose top-level node is an
`IterMapExpr`.
* [Affine] Implemented DetectPaddedIterMap
The existing DetectIterMap tries to rewrite index expression as a
linear combination of split/fused iterators, where the new iterators
cover the exact same indices as the original expression.
DetectPaddedIterMap relaxes this condition, allowing the new iterators
to cover a superset of indices that the initial index expression
covered. It uses the minimum amount of padding necessary to represent
these transformations, and also a predicate that identifies any
padding that has been added.
This is a utility function to be used for layout transformations of
buffers, in cases where the pre-transformation shape of the buffer
does not evenly fit into the post-transformation shape.
* [IndexMap] Implemented IndexMap::NonSurjectiveInverse
Allow non-surjective transformations, with DetectIterMap used to
determine the minimum padding to insert. Returns the inverse
function, along with a predicate that identifies padding indices. The
predicate is in terms of the transformed variables.
* [IndexMap] Exposed methods to python
- `IndexMap::Inverse` exposed as `IndexMap.inverse`
- `IndexMap::MapShape` exposed as `IndexMap.map_shape`
- `IndexMap::NonSurjectiveInverse` exposed as `IndexMap.non_surjective_inverse`
* [IndexMap] Extracted _assert_equal_index_map into class method
In preparation for adding additional tests for the IndexMap class,
which will require this functionality.
* [IndexMap] Added unit tests for new behavior
* Re-enabled divisibility check in CheckMapping
Initially disabled as dynamic shapes resulted in padded lengths whose
divisiblity couldn't be proven. Re-enabled along with a
simplification rule to resolve it.
* Fixed breakage in compute_at primitive
* Corrected typos/examples in docstring
We no longer run simulator automatically, so this is not necessary.
Also, the only way to pass options to the simulator was by setting
an environment variable. That variable (HEXAGON_SIM_ARGS) should
be set independently by the user from now on.
* Add Adreno GPU target and topi supporting textures
- There are 5 compute/schedules: conv2d for NCHW/NHWC, depthwise_conv2d
for NCHW/NHWC, average pooling
- Fix of dynamically allocated textures caching
- Add texture-nhwc scope
- Fix issue with codegen of vars having non acceptable symbols
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
* Address comments
* Add vectorization into some adreno pool flow
Co-authored-by: Li <quic_lih@quicinc.com>
* Fix adreno tests for running on the opencl host platform
* remove unnecessary kDriverVersion in DeviceAttrKind
* Move utils adreno functinos to separate shared file
* fix black hits
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
Co-authored-by: Li <quic_lih@quicinc.com>
* [microNPU] Added options to Cascader
* Added option to toggle multi-dimensional striping, it is disabled by
default because it has a very high computational cost. Single
dimension striping shares most of the benefit with greatly reduced
cost.
* Added multiple developer/debugging options prefixed with 'dev_'
Also added these options to tvmc.
* Added cascader logging, if enabled it will dump information about the
cascader proposals to a 'cascader_log.json' file.
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
Change-Id: I2ec59ae0bd84b73b2cc4bc56d39e3831b0aeec27
* Updated memory_reduction testcases
Also added enable_striping to plan_generator.h
Change-Id: I496b30ed6af6f0730087329cd81a69c5040a5e4d
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
This commit adds a cascader option to enable
striping explicitly.
When doing so fixed a bug that is associated
with block config selection, that will be
triggered when striping is disabled.
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
A number of small fixes and refactors to improve the robustness of
the TensorRT integration.
Co-authored-by: Mark Shields <mbs@octoml.ai>
Co-authored-by: Mark Shields <mbs@octoml.ai>
Refactor roofline_analysis to use a pass instrument to save TIR code
from compilation for feature extraction. This should support different
compilation pipelines and avoids recompiling the module twice.
* [microNPU] Add support for conv2d running on two cores on U65
The 512 mac variant has two cores that processes the weights in
parallel, so we need to split the weights and biases into two
and encode them separately.
Change-Id: I53791f614288ac4df181b9462fc632d35b934a86
* Changes due to rebase
* Rebase, improve DivideConstants and expand testing
Make the DivideConstants to operate on non-flattened
tensors to support two core execution in U65.
* Adds a script blocklint.sh that checks for non-inclusive words
* Updates the task_lint.sh script to call blocklint.sh
* Replaces the terms Master and Slave where possible
* Replaces the terms Blacklist and Whitelist
* Added pass that creates a semi-dynamic dispatcher around a relay module.
* Added automatic padding feature.
* Output slicing working.
* Multiple input support working i think.
* Added test file.
* Improve comments.
* Fix lint.
* Allow default values.
* Fix docstring.
* Improved documentation based on feedback.
* Add extra check for record loading.
* Improve variable names.
* Add type inference to make sure things worked.
* Added support for multiple outputs.
This reverts commit aa3bcd9d33, because it
fails on Windows CI as reported in issue #11220. PR #11223 tries to address
it but is is failing in the regular CI with testing issue on Hexagon.
Siyuan Feng