(See https://discuss.tvm.apache.org/t/byoc-supporting-cutlass-byoc-with-collage/12796/6 for
context, which in turn is part of Collage (https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md).
This adds a new 'DSO exportable' runtime module representing the contents of a .o file. It
allows external codegen toolchains to yield a result which:
- Like CSource modules, can be conveyed directly to the final export_library compilation
step for linking into the final .so and saved to a know location without risk the
underlying code artifact will be lost.
- Like DSOLibrary modules, are self contained so that no additional compile-time arguments
need be conveyed from the CSource module to the final export_library command line
Since this is the third flavor of 'DSO exportable' module, add a Module::IsDSOExportable.
Since adding the above, can't resist also adding a Module::ImplementsFunction virtual and
calling it from TEComplier to check if an external codegen function actually provided the
implementation it promised.
Note:
- I've left the existing implementation of runtime.load_module alone which
relinks .o files to .so files.
- Though also contained in the .o metadata, I require static libraries to always
carry their list of exported function names.
This is all pretty stop gap pending a good rework of TVM to supoprt the notion of artifacts
and, perhaps, build rules.
* [microNPU] Expose compute cycle annotations to TIR lowering
Adds an AttrSttmt "compute_cycles_hint" to each NPU operation for later
passes to consume.
Change-Id: I09779bdab6de6ef2094db610bb20d6e052e68ee3
* compute_cycles->compute_cycles_hint
Change-Id: Iebd71e699522e92a28fd321ffdb41ed7924db4e0
* add test to check annotations in compilation flow
Change-Id: Idcdcc8c8b5536c4732f297246b71aa8378a2732c
* add compute cycles hints for copy operations
Change-Id: I007ba19732e16081fa2ea9baca40c64a653c93cf
* fixing annotations for copies and improving test coverage
Change-Id: Ib812c4151fab03f4c1adcc016b4e798003a22e5e
* rebase
Change-Id: I653101908706096ae25ad1ebf08e7b6c4f1196c7
* refine the code style (#10112)
* support more data types in oneDNN BYOC
* consider dtype when query layout
* support more translation of blocked layout
* refine log for invalid layout transform
* reset N and C for the weights
* support multi-blocking in TransDims2Plain()
* add tests for bf16 oneDNN BYOC
* unregister 'round' OP in oneDNN BYOC
* restore the criteria for fp32 tests
* disable test_prune_dnnl_subgraph for bf16
* fix typo in dnnl.py
* delete tag::format_tag_last
* delete 'is_weight' in layout2tag()
* reuse dtype_dl2dnnl()
* fix lint errors
* change to WARNING for invalid laytout transform
* skip bf16 tests if AVX512 is unavailable
* [TIR][Schedule] Added Schedule.transform_layout_sugared
* [TE][TIR] Reduced duplication in TE/TIR layout transformations
Previously, the implementations of `tir.IndexMap.from_func` and
`te.Stage.transform_layout` had significant duplication to handle
argument parsing. This commit extracts the shared logic into
`tir.IndexMap`.
* Enabled *args in Schedule.transform_layout_sugared
* Fix lint error
* Allow Schedule.transform_layout_sugared to set axis separators
* Merged transform_layout_sugared functionality into transform_layout
* Fix lint errors
* Fix lint error
* Fixed docstring errors
* Updated/tested TransformatLayoutTraits::UnpackedAsPython
* Disabled exec-used check for running trace.as_python()
* Updated SetAxisSeparatorTraits::UnpackedAsPython
* Updated unit test that was added in merge commit
* Fixed the argument name for TensorizeTraits
This wasn't checked before, but was the only other issue caught by the
updates to verify_trace_roundtrip.
* Re-enable type checks of transform_layout/set_axis_separator
Disabled while waiting for https://github.com/apache/tvm/pull/11289,
which was required for the `Tuple` argument.
* Updated a few additional transform_layout usages from main
* Finish support for list-of-targets
This finishes the work started in https://github.com/apache/tvm/pull/11173 to support
'external codegen' targets in the N build-like API surfaces.
- It turns out it's ok if a build is given only a single 'external codegen' target, so remove that check
in CompilationConfig::Init. When Collage builds a 'candidate partition' it does so for a single target.
As far as Collage is concerned it does not care whether the target is regular (eg Target("cuda")), or
for a specific external codegen (eg Target("cutlass")), it just passes the target into the build.
- Add CompilationConfig::FindPrimitiveTargetForKind which I'll later need to retrieve
the external codegen Target instance corresponding to a "Compiler" attribute value.
- Target.update_target_host_consist was supporting three API styles:
- single target
- map from device type to target
- map from target to IRModule (for the ir_to_runtime API)
I replaced all those calls with a more specialized 'canonicalize' call:
- Target.canonicalize_target_and_host
- Target.canonicalize_multi_targets_and_host
- Target.canonicalize_target_map_and_host
In particular, all the tuning interfaces (task extraction, tuning, tuning records) all explicitly
*do not* support multiple targets since the underlying code just doesn't support that.
* - Lints
- Revert unintended changes
* - more lints
* - Fix model_library_format handling of target.
- Improve comments in compilation_config.h
* - Lints
- Update target/target_host params documentation
* - Fix micro library format tests
- Rev micro library format from 5 to 6
- Use Target.current() in a few places
* - eta contract comprehension
* - Woops, one more device: target map left
- Handle host already being in Target
* - lint
* - lint
* - Bug with append
- Take device type from target
* - Fix hexagon
* Fix int8 cuda kernels on older SM versions
* Update target.py
* Simplify initialiasation of do_tensorize
* Simplify initialization of do_tensorize dense
* Simplify initialization of do_tensorize in group_conv_nchw
* Fix tensorize for conv2d_int8 as well.
* Try to make linter happy
* make linter happy
* Fix wrong commit to auto_scheduler
The current meta-schedule uses a PostProc `RewriteUnboundBlock` to auto-bind blocks to threads. However, it's a post proc, which means there are no search opportunities, and always splits with `factor=1024`.
This PR adds a new search rule called `AutoBind` to do a similar thing to bind threads with sampled factors. Also with a corresponding mutator.
After applying this rule, we get some positive perf results (on RTX-3080):
Element-wise: from 2.76 us to 2.48 us
Conv2d Winograd: from 29.45 us to 18.96 us (ansor 22.00 us)
Resnet18: from 0.591 ms to 0.531 ms (ansor 0.565 ms)
* [Schedule] Allowed typing.Tuple in tir.schedule._type_checker
Previously, `typing.Tuple` annotations could not be used with
`tir.schedule._type_checker.type_checked` annotations. This allows
`Tuple` type annotations to be type-checked.
* Revert change, allow tuples input as List arguments
* Suppress mypy errors
Directly interacting with a type object would otherwise cause some
false positives.
* Corrected unit test for allowing tuples to be used as typing.List
* Represent multi-type lists as List[Union[...]] instead of List[Any]
This gives a better error message and plays nicely with _type2str,
since `typing.Any` doesn't have a `__name__` field.
* Fix function number datatype from char to uint16_t
rewrite the modified part to pass lint check
Use 2 bytes for func num in fun_registry
Fix errors in linter
Add the declaration of the helper functions
set 2 bytes for func num in func_registry test units
pass num_func by value
This commit change the datatype of the number of the function from 1 Byte to 2 Bytes.
Besides, I use some helper functions to access the number of function and the first function name.
* Fix aot_executor_module to unbreak CI.
* Fix GraphExecutorModule.
* Remove graph_json_to_c_func_registry.
* No longer needed and not called anywhere.
* Superseded by emitting the FuncRegistry directly in codegen.
Co-authored-by: 嚴中璟 <a1245967@gmail.com>
* [microNPU] Fix bug in channels extraction in the matcher
If the input tensor layout is in NHCWB16, we were passing W value
instead of the channels to get_valid_block_configs.
* Add test for conv2d
* [TFLite] Add support to int16 data type in TFLite frontend
Add support for int16 data type and int64 biases/accumulators in
the TFLite frontend.
Adjusts TFLite tests to cover int16 convolutions and element-wise;
Fixes a minor typo negtive->negative in the element-wise tests.
* Update src/relay/qnn/op/convolution.cc
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
* [TVMC][ETHOSN] Improve target string to avoid duplication
Improves the TVMC target string to avoid duplication of the
NPU variant. The new target string will require the just the NPU
name followed by -variant=n78. The old target string is deprecated
and will be removed in a subsequent version of TVM.
Change-Id: I4638f36788df3f478435ac13d3531aad2b23f204
* fix linting
Change-Id: I76a9da511899f24a163be669877605cd1a440022
* fix make variant functions and update test error message
Change-Id: Iff553d4b255c0ce0b86bad42eaa94ee9b1c62508
* [TVMScript] Support function call to help construct AST
* add test
* update test
* more comment
* fix for avoiding Buffer.vload(...) case
* update parse error msg
* wrap func call with try / catch, emit error msg
* silence pylint
* support ConvTranspose explicitly specified output_shape
* fix unit test case
* fix lint test
* retest
* fix code error
* fix lint test
* update test
* retest
* fix test onnx official tests
This PR allowed users to set logging level without giving a logger config. Previous implementation hard-coded `logging.INFO` as the default logging level and requires a logger config to change it. Now the logging level and handlers can be inherited from the current `tvm.meta_schedule` logger setting.
* add relay.f.frontend.fm_oneflow support cnns
* support cuda
* fix mobilenetv2 and reviews
* fix: model without meta info
* support eager and yolo, add test
* fix: license
* add: tutorials
* fix: support new graph
* fix some comments
* refine
* fix concat op convert bug
* refine
* refine
* change cuda to cpu
* fix bug
* fix ci error in tvm
* fix pylint check
* delete useless file
* add skimage package in docker
* fix ci error
* fix bug
* add oneflow fronted test in ci
* merge conflict
* fix tutorial
* try to find error in ci
* revert
* merge conflict
* black oneflow
* Delete from_oneflow.py
* restruct oneflow fronted
* support vision-transformer
* black format
* update black version and reformat
* fix ci error
* fix doc error
* fix gpu fronted test failed
Co-authored-by: hhhfccz <hjk1938927583@163.com>
* [ROOFLINE] Roofline analysis over RPC
Run roofline analysis on remote devices if requested. Peak flops and
peak bandwidth estimation are done on the remote device.
* allocate testing arrays directly on device and randomly fill
* forgot to include remote
* lower flops ratio, machine may be using multiple threads
* forgot fill
* [QNN] Enable constant folding for QNN operations.
This commit enables constant folding for QNN operations.
This functionalty is disabled by default, use fold_qnn=True to enable.
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [NFC] Fixed comments
* Added more unit tests for QNN opers in constant folding pass.
* Address PR feedbacks
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [Debug] Error logging in DetectIterMap
* [Affine] Allowed PrimExpr argument to NormalizeIterMapToExpr
This allows it to be used for any expression containing an
`IterMapExpr`, not just expressions whose top-level node is an
`IterMapExpr`.
* [Affine] Implemented DetectPaddedIterMap
The existing DetectIterMap tries to rewrite index expression as a
linear combination of split/fused iterators, where the new iterators
cover the exact same indices as the original expression.
DetectPaddedIterMap relaxes this condition, allowing the new iterators
to cover a superset of indices that the initial index expression
covered. It uses the minimum amount of padding necessary to represent
these transformations, and also a predicate that identifies any
padding that has been added.
This is a utility function to be used for layout transformations of
buffers, in cases where the pre-transformation shape of the buffer
does not evenly fit into the post-transformation shape.
* [IndexMap] Implemented IndexMap::NonSurjectiveInverse
Allow non-surjective transformations, with DetectIterMap used to
determine the minimum padding to insert. Returns the inverse
function, along with a predicate that identifies padding indices. The
predicate is in terms of the transformed variables.
* [IndexMap] Exposed methods to python
- `IndexMap::Inverse` exposed as `IndexMap.inverse`
- `IndexMap::MapShape` exposed as `IndexMap.map_shape`
- `IndexMap::NonSurjectiveInverse` exposed as `IndexMap.non_surjective_inverse`
* [IndexMap] Extracted _assert_equal_index_map into class method
In preparation for adding additional tests for the IndexMap class,
which will require this functionality.
* [IndexMap] Added unit tests for new behavior
* Re-enabled divisibility check in CheckMapping
Initially disabled as dynamic shapes resulted in padded lengths whose
divisiblity couldn't be proven. Re-enabled along with a
simplification rule to resolve it.
* Fixed breakage in compute_at primitive
* Corrected typos/examples in docstring
We no longer run simulator automatically, so this is not necessary.
Also, the only way to pass options to the simulator was by setting
an environment variable. That variable (HEXAGON_SIM_ARGS) should
be set independently by the user from now on.
* Add Adreno GPU target and topi supporting textures
- There are 5 compute/schedules: conv2d for NCHW/NHWC, depthwise_conv2d
for NCHW/NHWC, average pooling
- Fix of dynamically allocated textures caching
- Add texture-nhwc scope
- Fix issue with codegen of vars having non acceptable symbols
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
* Address comments
* Add vectorization into some adreno pool flow
Co-authored-by: Li <quic_lih@quicinc.com>
* Fix adreno tests for running on the opencl host platform
* remove unnecessary kDriverVersion in DeviceAttrKind
* Move utils adreno functinos to separate shared file
* fix black hits
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
Co-authored-by: Li <quic_lih@quicinc.com>
* [microNPU] Added options to Cascader
* Added option to toggle multi-dimensional striping, it is disabled by
default because it has a very high computational cost. Single
dimension striping shares most of the benefit with greatly reduced
cost.
* Added multiple developer/debugging options prefixed with 'dev_'
Also added these options to tvmc.
* Added cascader logging, if enabled it will dump information about the
cascader proposals to a 'cascader_log.json' file.
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
Change-Id: I2ec59ae0bd84b73b2cc4bc56d39e3831b0aeec27
* Updated memory_reduction testcases
Also added enable_striping to plan_generator.h
Change-Id: I496b30ed6af6f0730087329cd81a69c5040a5e4d
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
This commit adds a cascader option to enable
striping explicitly.
When doing so fixed a bug that is associated
with block config selection, that will be
triggered when striping is disabled.
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
Mark Shields