This PR allowed users to set logging level without giving a logger config. Previous implementation hard-coded `logging.INFO` as the default logging level and requires a logger config to change it. Now the logging level and handlers can be inherited from the current `tvm.meta_schedule` logger setting.
* add relay.f.frontend.fm_oneflow support cnns
* support cuda
* fix mobilenetv2 and reviews
* fix: model without meta info
* support eager and yolo, add test
* fix: license
* add: tutorials
* fix: support new graph
* fix some comments
* refine
* fix concat op convert bug
* refine
* refine
* change cuda to cpu
* fix bug
* fix ci error in tvm
* fix pylint check
* delete useless file
* add skimage package in docker
* fix ci error
* fix bug
* add oneflow fronted test in ci
* merge conflict
* fix tutorial
* try to find error in ci
* revert
* merge conflict
* black oneflow
* Delete from_oneflow.py
* restruct oneflow fronted
* support vision-transformer
* black format
* update black version and reformat
* fix ci error
* fix doc error
* fix gpu fronted test failed
Co-authored-by: hhhfccz <hjk1938927583@163.com>
* [ROOFLINE] Roofline analysis over RPC
Run roofline analysis on remote devices if requested. Peak flops and
peak bandwidth estimation are done on the remote device.
* allocate testing arrays directly on device and randomly fill
* forgot to include remote
* lower flops ratio, machine may be using multiple threads
* forgot fill
* [QNN] Enable constant folding for QNN operations.
This commit enables constant folding for QNN operations.
This functionalty is disabled by default, use fold_qnn=True to enable.
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [NFC] Fixed comments
* Added more unit tests for QNN opers in constant folding pass.
* Address PR feedbacks
Co-authored-by: Alexander Peskov <peskovnn@gmail.com>
* [Debug] Error logging in DetectIterMap
* [Affine] Allowed PrimExpr argument to NormalizeIterMapToExpr
This allows it to be used for any expression containing an
`IterMapExpr`, not just expressions whose top-level node is an
`IterMapExpr`.
* [Affine] Implemented DetectPaddedIterMap
The existing DetectIterMap tries to rewrite index expression as a
linear combination of split/fused iterators, where the new iterators
cover the exact same indices as the original expression.
DetectPaddedIterMap relaxes this condition, allowing the new iterators
to cover a superset of indices that the initial index expression
covered. It uses the minimum amount of padding necessary to represent
these transformations, and also a predicate that identifies any
padding that has been added.
This is a utility function to be used for layout transformations of
buffers, in cases where the pre-transformation shape of the buffer
does not evenly fit into the post-transformation shape.
* [IndexMap] Implemented IndexMap::NonSurjectiveInverse
Allow non-surjective transformations, with DetectIterMap used to
determine the minimum padding to insert. Returns the inverse
function, along with a predicate that identifies padding indices. The
predicate is in terms of the transformed variables.
* [IndexMap] Exposed methods to python
- `IndexMap::Inverse` exposed as `IndexMap.inverse`
- `IndexMap::MapShape` exposed as `IndexMap.map_shape`
- `IndexMap::NonSurjectiveInverse` exposed as `IndexMap.non_surjective_inverse`
* [IndexMap] Extracted _assert_equal_index_map into class method
In preparation for adding additional tests for the IndexMap class,
which will require this functionality.
* [IndexMap] Added unit tests for new behavior
* Re-enabled divisibility check in CheckMapping
Initially disabled as dynamic shapes resulted in padded lengths whose
divisiblity couldn't be proven. Re-enabled along with a
simplification rule to resolve it.
* Fixed breakage in compute_at primitive
* Corrected typos/examples in docstring
We no longer run simulator automatically, so this is not necessary.
Also, the only way to pass options to the simulator was by setting
an environment variable. That variable (HEXAGON_SIM_ARGS) should
be set independently by the user from now on.
* Add Adreno GPU target and topi supporting textures
- There are 5 compute/schedules: conv2d for NCHW/NHWC, depthwise_conv2d
for NCHW/NHWC, average pooling
- Fix of dynamically allocated textures caching
- Add texture-nhwc scope
- Fix issue with codegen of vars having non acceptable symbols
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
* Address comments
* Add vectorization into some adreno pool flow
Co-authored-by: Li <quic_lih@quicinc.com>
* Fix adreno tests for running on the opencl host platform
* remove unnecessary kDriverVersion in DeviceAttrKind
* Move utils adreno functinos to separate shared file
* fix black hits
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
Co-authored-by: Egor Churaev <egor.churaev@gmail.com>
Co-authored-by: Li <quic_lih@quicinc.com>
* [microNPU] Added options to Cascader
* Added option to toggle multi-dimensional striping, it is disabled by
default because it has a very high computational cost. Single
dimension striping shares most of the benefit with greatly reduced
cost.
* Added multiple developer/debugging options prefixed with 'dev_'
Also added these options to tvmc.
* Added cascader logging, if enabled it will dump information about the
cascader proposals to a 'cascader_log.json' file.
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
Change-Id: I2ec59ae0bd84b73b2cc4bc56d39e3831b0aeec27
* Updated memory_reduction testcases
Also added enable_striping to plan_generator.h
Change-Id: I496b30ed6af6f0730087329cd81a69c5040a5e4d
Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>
This commit adds a cascader option to enable
striping explicitly.
When doing so fixed a bug that is associated
with block config selection, that will be
triggered when striping is disabled.
Co-authored-by: Elen Kalda <elen.kalda@arm.com>
A number of small fixes and refactors to improve the robustness of
the TensorRT integration.
Co-authored-by: Mark Shields <mbs@octoml.ai>
Co-authored-by: Mark Shields <mbs@octoml.ai>
Refactor roofline_analysis to use a pass instrument to save TIR code
from compilation for feature extraction. This should support different
compilation pipelines and avoids recompiling the module twice.
* [microNPU] Add support for conv2d running on two cores on U65
The 512 mac variant has two cores that processes the weights in
parallel, so we need to split the weights and biases into two
and encode them separately.
Change-Id: I53791f614288ac4df181b9462fc632d35b934a86
* Changes due to rebase
* Rebase, improve DivideConstants and expand testing
Make the DivideConstants to operate on non-flattened
tensors to support two core execution in U65.
* Adds a script blocklint.sh that checks for non-inclusive words
* Updates the task_lint.sh script to call blocklint.sh
* Replaces the terms Master and Slave where possible
* Replaces the terms Blacklist and Whitelist
* Added pass that creates a semi-dynamic dispatcher around a relay module.
* Added automatic padding feature.
* Output slicing working.
* Multiple input support working i think.
* Added test file.
* Improve comments.
* Fix lint.
* Allow default values.
* Fix docstring.
* Improved documentation based on feedback.
* Add extra check for record loading.
* Improve variable names.
* Add type inference to make sure things worked.
* Added support for multiple outputs.
This reverts commit aa3bcd9d33, because it
fails on Windows CI as reported in issue #11220. PR #11223 tries to address
it but is is failing in the regular CI with testing issue on Hexagon.
* [Relay] Support 'external codegen targets'.
(Part of Collage, https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md)
This change prepares the VM and Relay target handling machinery to support
external codegen targets in addition to 'regular' targets. This allows us
to configure the build with Collage as follows:
```
host_target = tvm.target.Target("llvm")
targets = [tvm.target.Target("cuda", host_target),
tvm.target.Target("cutlass", host_target),
tvm.target.Target("cudnn", host_target)]
with tvm.transform.PassContext(...):
exe = tvm.relay.vm.compile(module, target=targets)
```
Four changes are required:
1. I introduce four new target kinds for the external codegens currently supported
by Collage. Others can be added as they are vetted for use by Collage. These
are given a device type matching the external codegen's assumption (ie just CUDA
currently), and given a target kind attribute "is_external_codegen" of True. The
latter is needed by Collage to signal the target kind name represents and external
codegen 'compiler' name. See the RFC for specifics.
2. I introduce the binary relation Target::IsExternalCodegenFor so that
external codegen targets can be related back to the 'underlying' targets
they are implicitly using in their codegen.
3. I rework the VMCompiler and BuildModule interfaces to accept an Array<Target> of
'raw targets' instead of a Map<Integer, Target>. This more general representation
is needed because we may now have multiple targets of the same device type
active simultaneously. I add new static methods on the Python Target to
convert to this form in a way that mimics check_and_update_host_consist.
4. I rework CompilationConfig to work from Array<Target> directly, to not depend
on the host_target argument (since dealt with on the Python side), and to
understand that if we have two targets for the same device type the non-external
codegen target takes precedence.
The change to CompilationConfig seems neutral with respect to the recent discussions
on compilation configuration representation and tvmc.
I made a few attempts to remove Target.check_and_update_host_const entirely in favor
of using CompilationConfig as the definitive target handling choke point but backed
out once they became too large.
* - Working on unit tests
* - Fix two Debug-only failures
* - Use Array<Target> in GraphExecutorCodegen/AOTExecutorCodegen ifaces instead
of CompilationConfig (don't want to bake it into any official APIs).
- Started unit tests.
* - Lints
* - Moar Lints
* - Fix some unit tests
* - Fix last unit test failures
* - whitespace
* - Address Eric's comments.
CI likely to fail due to stricter FindPrimitiveTargetOrFail but let's see.
* - Comment adjustments.
- Unit test for new Target members.
`tvm.analysis.roofline_analysis` adds estimated roofline performance to a
profiling report. The roofline model measures how close an operator gets
to best possible memory bandwidth or FLOP/s depending on whether it is
memory or compute bound. This computation uses the runtime of the
operator along with two numbers extracted from the TIR code: bytes of
memory touched and number of floating point operations. Because these
numbers are extracted from TIR, they may not be 100% accurate. The best
possible memory bandwidth and FLOP/s are measured by running small
programs that are memory and compute bound respectively.
For now, this function only works with llvm cpu targets, but it should
be possible to extend to GPU targets.
* Add test for registered scheduales - depthwise_conv2d
* added more test to depthwise_conv2
* adding new line at the end of the file
* reformatted the file
* resolve comments
* add schedule and tests for conv2d_transpose_nchw
* registering conv2d_transpose strategy and clean up test
* [Runtime][PipelineExecutor] Refactor PipelineExecutor.py add cross
compile support for pipeline executor.
Current pipeline_executor and pipeline_executor_build stay in same
file, this caused that the the running of pipeline_executor need support
from tvm and relay that is not available on edge device in which a runtime
library only can get build.
Pipeline executor used PipelineExecutorFactory to store the pipeline
configuration and export the pipeline executor library, but the current
export not support the cross compile, add related logic.
* fix ci issue.
* use runtime to replace relay and leave the export_library in
pipeline_executor.py.
* [microNPU] Match requantize in min/max with activation pattern
Optimizes a corner case where min/max + clip also produces a requantize
operation. Previously the requantize was lowered separately as an
identity operation which is unnecessary. Now the quantization parameters
from requantize will be used by the lowered min/max operation.
Change-Id: Id740d975bd8ba2952f3444ce1061acef560d74d7
* add random seed to legalization test
Change-Id: Ic4ff78af94c3e8250dba8e3ce5c2775fcc7a17f6
* hitting bug while running the reshape unit test. currently trying to reproduce error in script
* unit test passes
* ran make format
* removed print statements
* edited commentary
* moved the zero check outside of the ravel unravel and into the topi reshape defn
* ran cpplint
* changes from andrews comments
* derp
* black
* ran black on test_forward.py
* fixed test expected output
* retriggering CI due to hexagon test failure
* [TIR] Get read/write access precisely for opaque access.
When the opaque access is wrapped with tvm_access_ptr, we can get the access_mask
from tvm_access_ptr in BlockReadWriteDetector and put this opaque access to read_regions
or write_regions according to access_mask.
* [TIR] Add parameter extent for access_ptr.
Co-authored-by: sqing <qing.siqi@intellif.com>
* [CMSIS-NN] Moved TFLite model making to common area
Change-Id: Ic4dbc1919ff0b481c05daf7e57cf9b055c714c9c
* Fixed lint issues with tensorflow import
Change-Id: I7a520beec9c244e9c790d3e82733c2fb476f7e5e
* Resolved merge conflict with main
Change-Id: Iefe58dd321efae6eae26cd54a31c5923d0f1e32b
* Made TFLite layer creation explicit
Change-Id: I7fbf6a5a2163c1fada49477f86d84f1bc09bd57c
* Lint fix: added a missing docstring
Change-Id: If1fb8bb09c538c04e333ccab65a20cff247a504d
* [Metaschedule] Auto-tensorization for CPU / GPU dot product
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
* doc update
* add vnni conv2d test
* add dp4a test
* adding tests for rewrite_tensorize
* add rewrite_tensorize test
* add missing pydoc
* black
* more doc
* adding auto tensorize integration test
* add dp4a test
* fix target name
* fix dtype in test
* skip bert test
* replace hard-coded llvm intrinsic id in test with look up
* remove unnecessary include, add doc for the rest of params
* update postproc.h
* update doc
* fix shape in te matmul workload
* fix newline in cppdoc
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
* [Python] Populate setuptools description with README.md
Adds the description metadata for the setuptools descriptor file
`setup.py` with the contents of our existing README.md, which is
a common practice.
* Update python/setup.py
Co-authored-by: driazati <9407960+driazati@users.noreply.github.com>
* Update python/setup.py
Co-authored-by: driazati <9407960+driazati@users.noreply.github.com>
* Import pathlib and apply black formats.
Co-authored-by: driazati <9407960+driazati@users.noreply.github.com>
Margaret Qian