* Remove warning from Teensy boards
* Use a real timeout
* Skip assertion of whether a functional schedule exists
* Don't specify least significant digits for Teensy boards
This PR addes:
- All ExprDoc subclasses
- Their Python bindings
- Support of ExprDoc in PythonDocPrinter
- Unit tests for ExprDoc in PythonDocPrinter
Tracking issue: https://github.com/apache/tvm/issues/11912
This updates the version numbers after the v0.9.0 release and adds a version selector option for the v0.9.0 docs.
Co-authored-by: driazati <driazati@users.noreply.github.com>
* [TVMC] Workspace Pools Parameters
Attributes from tvmc are now passable into the created PoolInfo objects
inside WorkspaceMemoryPools. This is passed in to relay.build that get
attached to IRModule attribute.
* [TVMC] Workspace Pools Parameters
Address comments, fix linting. Testing improved.
Change-Id: Iea79329b6b9ec1cbc51e5c293449bf6dd43b00c5
* [TVMC] Workspace Pools Parameters
Update workspace pools test naming
Change-Id: Ib698d6248be1e6f44340f27db3641c985bc5c5d8
* [TVMC] Workspace Pools Parameters
Add test for parameter overrides.
Change-Id: I67d5470dcfbfbc9ab27f34e20a9269d2070193ca
* [TVMC] Workspace Pools Parameters
Rebasing over #10189
Updates to the way a WorkspaceMemoryPool object is created
Change-Id: I1f0e1d240343af311ddb3ed5c564cc1ab329f463
* [TVMC] Workspace Pools Parameters
Fix linting, fix CI
Change-Id: If75f8709ac4ad925655eca54b3e5c1bb09d025e8
* [TVMC] Workspace Pools Parameters
Add mcpu and mattr to target registry for cmsis-nn
Change-Id: I15257b8d01624c071c738cab6d12ecb84ed6cb16
* [TVMC] Workspace Pools Parameters
Added test for override on single pool when multiple pools are present
Updated functionality of parsing multiple attributes
Change-Id: I2c0745051b7a923dd7f75040bfb89bbc99376a11
Some operations were being offloaded when they are not supported
by the NPU, for example mean could get offloaded with different
quantization parameters for the input and output which is not
supported. Consequently, this meant that there would be a failure
during compilation or an output mismatch at runtime. Fixing this by
supplying the output information to the issupported checks that
determine whether an operation should be offloaded.
Change-Id: I8896f83dad3d1c837fbb85bf2836fc9325f9dec9
* Add support for relu slice op.
* Format code
* removing out_shape in relu def and lint issues
* removing out_shape in relu def and lint issues
* Changes as per the new format
Co-authored-by: Venkat Rasagna Komatireddy <89959097+rasagna-quic@users.noreply.github.com>
Co-authored-by: Venkat Rasagna Reddy Komatireddy <rasagna@hu-rasagna-hyd.qualcomm.com>
* Make Arduino API server obey timeout
* Pass arm_cpu as default option to micro testing
Syntax fix
Increase Zephyr default stack size for create_aot_session
* Set write_timeout when appropriate
* Fix unit tests and linting
Check whether arm-cpu flag is breaking tests
Update tests for arm-cpu flag
* TVMC: Add new text/relay frontend
This feature enables passing a textural representation of a relay module to the tvmc command line.
Example: `tvmc compile relay.txt --target c --runtime=crt --executor=aot --executor-aot-unpacked-api=1 --pass-config tir.disable_vectorize=1 -f mlf`
Currently it is not possible to supply parameters as it is mainly intended to be used for testing certain relay functions or operators. In the future (with minor changes to the tvmc frontend api) params could be passed via an additional i.e. `params.bin` file
This commit also adds minimal unit testing of the added feature.
Resolve PR comments
TVMC: add warning if relay frontend is used
* [TVMC] populate parameters with random values instead of ones
* [TVMC] Relay frontend: do not populate input tensor buffers if --input-shapes is provided
This prevents that the constants inputs are used for Constant folding,
thus changing the complexity of the model.
If there would be a way, to distinguish between model inputs and parameter this
workaround would not be required.
* [TVMC] Relay frontend: check provided file contents before calling tvm.parser.fromtext()
This adds a generated property "features" to the `Target` which can
contain a read-only list of available features in line with
https://github.com/apache/tvm-rfcs/pull/78.
Features are re-generated upon parsing into a `Target` object rather than being
attached as `attrs`. The `Target` JSON is therefore stored without the
inferred `features` attached.
* [Texture] Add 2d memory support into static memory planner
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
* Add test verifying GraphPlanMemory work for 2d memory
Co-authored-by: Chris Sullivan <csullivan@octoml.ai>
* [Relay] Allow partial virtual device annotations.
Previously CompilationConfig::CanonicalVirtualDevice required
the argument virtual device to contain a device type. However
now that virtual devices may contain memory scopes that's
unnecessarily strict.
With this change it is possible to write virtual device
annotations with just memory scopes, and let PlanDevices
flow those constraints along with the usual device constraints.
* - Make sure CanonicalVirtualDevice reuses FullyUnconstrained
* [Collage] PruneCandidates and demo_collage_partition.py
See https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md.
This completes our checkin of our Collage 'sketch' branch into main. Special thanks
to Matthew Barrett for his help getting this over the line.
The only C++ functionality added here is for 'pruning' candidates. This is a somewhat
speculative algorithm (and I've called that out in the comments) which tries to
elide candidate partitions which will 'obviously' not contribute to the final optimal
partitioning. For largish models such as GPT2 this can significantly reduce the number of
candidates we need to actually measure latency on. I beefed up the MockCostEstimator to
make it possible to assert pruning occured from within the test_pass_collage_partition.py
unit test.
The rest of this PR adds the demo_collage_partition.py driver file we've been using
to test and measure perfomance differences against various baseline (though only
for the CUDA ecosystem). To eliminate loading time the models of interest are directly
expressed in Relay text form in menangerie.py.
* - lint
* [Collage] CollagePartition pass
See https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md.
This adds the main CollagePartition pass, which:
1. Inspects all the targets in the CompilationConfig and builds
PartitionSpecs describing how to generate speculative CandidatePartitions
for them.
2. Runs the above rules on the model to collect all the candidates.
3. Eliminates candidates whose target contradicts any constraints already
imposed by, eg, device planning.
4. Eagerly estimates the cost of each candidate.
5. Performs a shortest path search to chose an 'optimal' set of candidate
partitions so as to minimize estimated model latency, such that every sub-expression
node is contained in exactly one candidate partition.
6. Coalesces adjacent optimal candidates which ended up on the same target.
7. Rewrites the model according to the chosen optimal partitioning.
As for the existing partition_for_<external codegen name> methods, the result of
CollagePartition can then be built using regular TVM.
Very special thanks to @mbaret for authoring test_pass_collage_partition.py.
Logic to prune the candidates after step 3 will be in a follow up PR since it
deserves its own testing. A demonstration driver will also come as a follow up.
* - lints
* - more lints
* - use the _ffi_api properly
Motivation:
Same IR node object can be referenced in several different contexts inside a larger IR object. For example, a variable could be referenced in several statements within a block.
This makes it impossible to use an object pointer to uniquely identify a "location" within the larger IR object for error reporting purposes. The `ObjectPath` class addresses this problem by serving as a unique "locator".
Tracking issue: https://github.com/apache/tvm/issues/11912
* [Relay] Move TOpPattern registration for nn.* to C++
Some of the Collage machinery is best tested from C++, but
requires Relay ops to have their "TOpPattern" registered.
However since the nn.* ops register on the Python side tests
can't rely on those ops.
The easy fix is to just move the registration to the
RELAY_REGISTER_OP block. However since kOpaque is the
default I did not preserve those registrations.
There's still a few dozen more exotic ops still registered
on the Python side. I've left them be.
* - D'oh! Even kOpaque ops must be registered.
Egor Churaev