* [Relay] Fix output dtype for conv2d wgrad when the original one is void
* fix cpplint
* also add out dtype information to dgrad
* also use out_dtype for wgrad
* remove redundant import
* Add memory pools to Metadata classes.
* Move ShapeToJSON to utils.
* Track returned TensorType from AOTExecutorCodegen.
* Support calling Relay functions with Tuple.
* Expand supported TIR calling conventions to work with C++ runtime.
* Rename MetadataModule to ConstLoaderModule.
* Add runtime AOT executor module.
* Add AOT code-generation.
* Add a runtime Module to mux between .text Metadata and live Metadata.
* Move launch_param to namespace
* Add test of c++ AOT.
* Fix incongruity between kTvmRuntimeCrt constant
* Expand ExecutorCodegenMetadata to include AOT runtime metadata.
* commit cpp test
* Make Metadata compile under C.
* Ignore ephemeral metadata_module export_model_library_format.
* This module does not need to be exported, since it is merely a C++
wrapper around get_c_metadata, and get_metadata is not used in C.
* address manupa, kparszsyc, masahi comments.
* further address comments
* clang and python format
* Fix broken test
* Address lingering comments from masahi, kparszyzc
Setting ANDROID_SERIAL_NUMBER=simulator will execute the tests on
simulator instead of a hardware device.
This patch also introduces an environment variable HEXAGON_RPC_LIB_DIR
to specify the location of the hexagon_api binaries. If unset, the
code will look for the binaries in the same way as before this patch.
1. Modify debug output to make it more readable
3. Replace magic number with a variable `error_ct_threshold`
3. Add function to set error counter threshold externally for debug purposes
A quick fix of the parser issue mentioned in #10327 .
Ranges and loops require `start` and `stop` to be PrimExpr, however, `BufferSlice` is not always scalar so it's not a `PrimExpr`.
This PR performs the transformation.
* set_input_with_index was implemented for VM
* clean code
* add getInputIndexFromName. add function descriptions. lint fix
* fix lint
* transfer comparison of parameter names number and assigned devices number to VMFunction constructor
* add GetVMFunctionWithName to Executable API
* clean code
* add SetInputWithName (set_input_with_name) to VM API
* join SetInputWithIndex and SetInputWithName to SetOneInputTensor (set_one_input) to VM API, the joined methods were removed
* fix lint
* some fixes after review
* add set_one_input method to python API of VirtualMachine
* pytests for set_input and set_one_input methods of VirtualMachine were implemented and checked
* CI restart
* construct simple model for pytests by relay instead of onnx tools (need for correct CI)
Co-authored-by: Valery Chernov <valery.chernov@deelvin.com>
Adding a few tests to confirm memory usage
with and without USMP.
- Supporting the toggle to disable storage_rewrite.
- There is a slight change to tir_to_cs_translator to
add index of Load nodes associated with NpuAddressRange objects
This is the C++ code for running Hexagon code on simulator via the
RPC mechanism. It is intended to be integrated into the current
HexagonLauncher, although the integration will require further changes
to the launcher python code.
The final goal is to be able to run the same file.py on either
hardware or simulator without needing to edit the python file, but
simply by changing the configuration of the execution platform
(i.e. something like --exectute-on=simulator as a command line or
in an environment variable). The exact details are still to be
determined.
* [Runtime][Pipeline Executor] multiple threads management and the
data forwarding notification mechanism.
In this patch we create working threads for each runtime of pipeline.
the threads would be terminated once the runtime class gets destroyed.
We also add a notification mechanism derived from the 'binding configuration'
of the runtime to forward the data notification.
* address review comments.
* address review comments.
* fix typo.
* fix typo.
* trigger build.
* address review comments.
* address review comments.
* address review comments.
* address review comments.
* [microNPU][4] Add the cascader Proposal generator
The Proposal generator takes optimal Plans and combines
them to find optimal 'Proposals' - sets of disjoint
Plans that cover every Part in a CascaderGraph. It
ultimately produces a Pareto-frontier of 'optimal'
Proposals in terms of estimated cycles and memory usage.
Change-Id: Id42099819a596496a5769bae22f08eeb75ec69b6
* Fixes
Change-Id: I4f5f2a298bd3bb379c7c8d179150358923b0dd66
This was broken in #10267, not sure how that commit passed CI (maybe some logic to figure out the PR diff in pylint is broken).
Co-authored-by: driazati <driazati@users.noreply.github.com>
* refactored GraphProto.from_onnx into smaller functions
* black formatted file
* removed line that does not seem to make sense. Is there a purpose that I missed?
* just to trigger CI pipeline
* add test
* compute added
* schedule works
* reuse dense_vnni schedule
* try an alternative approach to scheduling layout transform
* introduce a tunable knob to decide if compute_root
* check transpose condition
* support s8 + s8 input
* pylint
* [ARM_CPU] Conv2d int8 intrinsic for cortex-A72
Add an intrinsic that performs a dot product of 8 4-element vectors at
once. Also conditionally inline fused operators into the main
convolution loop depending on convolutions size. Small convolution = no
inlining. Performance improves by ~20% on mobilenet on raspberry pi 4
and ~30% improvement on performance for the individual convolutions.
* ignore incorrect lints
* fixup fstring
* revert changes to conv2d_NCHWc (not int8)
* remove error check, apparently tests rely on it
* refactor alter op layout
* [TIR] Introduce tir.allocate_const to TIR
This PR is adding non-scalar constant representation in TIR. This is used to
express constants (i.e., parameters) in the TIR instead of bypassing the
TIR as it's done until now.
Change-Id: Id3afc4d7197260cb43ecde60f05ccbce3fc42430
Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com>
Change-Id: Id4a09a637c9c1fd7d49989c6c10f474a78569e18
* [TIR] Integrate tir constant nodes in compilation pipeline
This PR integrates tir.allocate_const to the compilation pipeline to support --link-params.
Change-Id: Ic8d0cb75d596299fcae7078b304598afbf0c5494
Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com>
Change-Id: Id98cc682bbfacfe75c4d8b260fd41658f1f196b2
* [TIR] tir.const extraction
This commit tries to implement an amendment to tir.constant RFC
with centralized storage of constant data within the IRModule
Please note that data and irmod_storage_idx are not mutual exclisive
further more the irmod_storage_idx is valid only immediatly after
prim func addition to the mod or after update within the mod.
If prim func is out of the the module scope then the index become
meangless. irmod_storage_idx also is not used in calculation of hash
function of the tir.constant node.
Change-Id: I40742ed580468b0252ea3fec02184cba65e20871
* unit test fixed
Change-Id: Ied2186554d4cbad44b2346216c8be92449e55732
* cmsis-nn codegen fix
Now handled case when params of the functions came as constants
Change-Id: I5874e182e34ef94e23048eaf3c61b01a56d91131
* Fixes for unittests
Change-Id: I5b82ee3f80337155706b5470973f494a301b5d90
* Rebasing tests fixes
Change-Id: I94ac87907081bab53c1dd1ab2db106ae057b4b19
* Linter: added method param description
Change-Id: I2f8c4c8d244b74c794abaa6079c46cc593ffcbdb
* Printing removal fix
This patch removes forgotten print in fuse_ops
Change-Id: I4bb5934f3b4cd5fde19d36a8e3319aae136bce8a
* Bugfix
Fixed concurrent map update bug here
Change-Id: Ifec3bf5030086d9079b9e493096f17dfd82297ec
* Reworked logic for not to introduce empty constant list to modue attrs
Change-Id: I082c85b3b4b70c218f0d714f5613ef6e178bd020
* Added support for tir builtin::tvm_access_ptr
This fixed unit tests for tests/python/integration/test_arm_mprofile_dsp.py
Change-Id: I10919f301ef9ddc3fd87f0e1a8414e9a52fc7938
* Unit test fix
Fixes unit tests in torch frontend
Change-Id: I6c179834f93dd202605d1ce5a7f07d987b9dc469
* Addressed requested changes
Addressed changes requested upstream
Change-Id: I741e52b89eb285732c23b1ac7ff277e757a088c3
* Namespace usage changed to conform earlier C++ standard
Change-Id: I1b29238cfe2a6bedb525f4f823a3a540f631d836
* Bugfix
Change-Id: I57a44b714b307278a243817ec2864e53ad31366b
* updated IRModuleNode::ExtractPrimFuncConstants
Updated IRModuleNode::ExtractPrimFuncConstants as per
request upstream.
Change-Id: I35db0145fb5827efd0445ce665d0c99465274016
* Minor changes
typo fixd
renamed ExtractPrimFuncConstants to ExtractConstants
removed getters/setters from FuseMutator and added parametrized
constructor
Change-Id: Ib2326805781779b88c963a8642ff683c8755956e
* Moved LinkedParam/LinkedParamNode
Moved LinkedParam/LinkedParamNode from tvm::tir namespace to tvm
namespace
Change-Id: Ie3f0303bd4f7890c6d680268c91f2051977bc7f4
* Addressed upstream comments
Changed BindParams argument to Array<NDArray>
Removed 'name' argument from te.const
Switched to in-depth comparision of NDArrays in constant de-duplication
Removed extra final comma from NDArrayToTIR
Changed return type of ConstantAllocationSize to int64_t
Made link_param a tvm.testing.parameter for test_fuse_take and test_fuse_gather_nd
Change-Id: I4285099cc63756aa5ebe91a5bd207d4135499b41
* Removed unnecessary forward declaration
+linter
Change-Id: I2a6c0d1f97773aeb1ae3f458da252a22079ccdb1
* Constant extractor now is a separate pass
Change-Id: Ia4adca9d3315b26fbdc006ef7c115900c081e303
* Added forgotten file + unit test fix
Change-Id: Ice305f4fefd13fe95e97574e6d63ffeb664621df
* Changed to IRModule pass
Refactored ExtractPrimFuncConstants to IRModule pass.
deDup -> DeDup
Refactored logic of Applicator supplementary class
Change-Id: I6c120d175eb6790ba90f176c4f856bde8f0c7c94
* bugfix after rebasing
Change-Id: Ie3ee6ea2479476a30f486baef74f20070f117942
* -v -> -vv to have more debug information
Change-Id: I12c63731663b9c9ea574b9ed5cb17311ba3cf701
Co-authored-by: Giuseppe Rossini <giuseppe.rossini@arm.com>
* RelayViz interface and terminal ast-dump.
This PR follows https://github.com/apache/tvm/pull/8668, with splitting
out interfaces class and terminal ast-dump implementation.
This visualizer is aimed for quick look-then-fix, so the interface is
simple. Despite that, customization is still possbile through
implementing interfaces defined in `interface.py` or overriding existent
implementations inside a renderer module, like `terminal.py`.
A tutorial is also provided in this PR.
A graphviz renderer will also be contributed after this PR.
* lint and typo
We found this while converting an RNN model.
The relay tflite frontend use squeeze at converting unpack, but when the
unpack.axis=0, `None` is passed to relay.squeeze(), which would squeeze
all dimensions with length 1, causing different results from TFLite.
A possible fix might be, assign the unpack.axis as-is to relay.squeeze()
As for stridedslice, when the tflite frontend handles shrink_axis_mask,
the wrapped `begin` should be used, instead of the original one which
can be negative. It can cause errors at
https://github.com/apache/tvm/blob/d65ff6594d4d6db0062537a1d43c0504173b8e5c/include/tvm/topi/detail/strided_slice.h#L140
Related cases are also added to the python test.
* [TOPI] Add support for groupped conv3d
Change conv3d to use generic conv implementation which supports groupped
convolutions. Also, remove support for non-float16 tensorcore operations
as they cause large degradation in accuracy. Generic conv now supports
autoscheduler.
* correct none check
* add tests for floordiv simplification
* fixed incorrect test for autoscheduler
* formatting
* add groups to winograd
* fix tensorcore
* manually simplify index instead of relying on simplifier
* formatting
* add groups argument to conv3d_ncdhw_winograd_without_weight_transform
* formatting
* [UnitTest] Disable ptx mma tests on unsupported nvcc versions.
- Modified `tvm.contrib.nvcc.get_cuda_version` to return a
`(major,minor,release)` tuple rather than a float.
- Implemented `tvm.testing.requries_nvcc_version` decorator to specify
the minimum `(major,minor,release)` version needed to run a unit
test.
- Applied decorated to unit tests in `test_tir_ptx_mma.py` that fail
on earlier nvcc versions.
* Fix lint errors.
* Updated a few of the cuda version checks.
* More lint fixes.
* Only compare major/minor in find_libdevice, not release version.
Gustavo Romero