* [PROFILER] Add CSV output to profiler
This patch changes the profiler output from a string to a Report object.
A Report can either output CSV or the usual human-readable table.
* no spaces after commas
* Update src/runtime/profiling.cc
Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>
* fix gcc
* fix test
* overall percent fix
* rename overall -> device_metrics
Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>
* [Profiling,VM] Profiling interface for VM and Graph Runtime
* lint
* fix test
* make profiling test actually run
* Try to better match the graph runtime function names to vm
* formatting
* DurationNode.value -> microseconds; PercentNode.value -> percent; make frame sorting optional.
* renaming for the tvmcontext -> device change
* formatting
* remove old vm profiler get_stat api
* fix tests
* Get basic verison of VM RPC working
* Test case passes
* Clean up PR
* Lint
* Format
* Address Andrew R and TK feedback
* Add comment for Andrew
* Address Zhi's comment
* Format
* Fix broken test
* Some docstring fixes.
* Couple of small fixes:
- Use `west attach` instead of `west debug` in commandline to prevent
debugger from resetting device.
- Fix warning on use of led_pin in zephyr-runtime/src/main.c.
* Adding Zephyr demo runtime.
* Cleanup of uTVM tests and demo runtime.
* Working on QEMU support.
Need to add board-specific prj.conf files.
* Adding board-specific prj.conf files.
* Some cleanup.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Adding data for unit tests.
* Cleanup demo_runtime code.
* Fix up tutorial.
* Couple of small fixes:
- Use `west attach` instead of `west debug` in commandline to prevent
debugger from resetting device.
- Fix warning on use of led_pin in zephyr-runtime/src/main.c.
* Adding Zephyr demo runtime.
* Cleanup of uTVM tests and demo runtime.
* Working on QEMU support.
Need to add board-specific prj.conf files.
* Adding board-specific prj.conf files.
* Some cleanup.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Adding data for unit tests.
* Cleanup demo_runtime code.
* Fix up tutorial.
* Fix tutorial.
* Fix tutorial and runtime.
* Fix merge conflicts.
* Fix merge conflict.
* Remove redundant files.
* Revert dep.
* Fixup
* Add new files to check_file_type.py.
* Adding missing ONNX file.
* Fixup docs.
* Fix linting rule.
* small fixes
* Add missing file to check_file_type.py.
* clang-format this file.
* Fix formatting.
* Black formatting.
* Lint comments.
* Fix path for test.
* Bump CI.
* Update from_onnx.
* fix path
* Fixing
* Revert dmlc-core to 21cc7de0dc9fd6acb796e1be6181fa8e6b6c8f41
* Fix path again.
* Fix tutorial to not use actual Zephyr.
* Revert submodule version change
* Fix bad merge.
* Trying to fix this mess.
* Fix formatting.
* context -> device
* Removing tutorial since I can't get it to pass CI.
Co-authored-by: Mehrdad Hessar <mehrdad.hessar@gmail.com>
Co-authored-by: Andrew Reusch <areusch@octoml.ai>
* Introduce Model Library Format export format.
* This function produces a stable on-disk representation of TVM's
compiler output.
* It's intended just for use with the C runtime for microTVM right
now. It could be expanded for other use cases.
* This PR implements the Model Library Format RFC, which ultimately
is intended to support the Project Generator API (RFC
forthcoming).
* There may be some changes to the format without revving the version
number until downstream consumers are known. The Project Generator
API is the first such known downstream consumer.
* There are no plans currently to support generating old Model
Library Format from TVM. The version number is intended as a
compatibility check between the generator and downstream consumers.
* [RUNTIME] Move Map into runtime
This allows us to use Map to store parameters needed at runtime.
* node.{Array|Map} -> runtime.{Array|Map}
* missed some renames
* Fix stack overflow when partially-__init__ Node raises exception.
* If a Node subclass raises an exception and ctypes is in use before
__init_handle_by_constructor__ is called (or self.handle is
otherwise set), a Python stack overflow could result. This is
because the unset handle slot causes self.handle accesses to
fallback on the getattr(self, 'handle') method, invoking
NodeGetAttr.
* Then I believe this causes an infinite loop.
* The fix is to make Node.__getattr__ raise AttributeError for all
attributes in __slots__, then make __del__ tolerant to missing
self.handle.
* I don't believe cython is affected because it implements a
descriptor to access its underlying chandle and that shouldn't be unset.
* black format
* actually use handle instead of self.handle
* Created CSourceMetaData module for model metadata
* Currently, there is a MetaData module to capture constants
conditionaly if the runtime modules implement const init
PackedFuncs. However, this one relies on a load process
in which the metadata is created on volatile memory that
may be not usable in uTVM environments.
* There is a need for model level metadata that is valid
across all runtime modules such as the func registry
when creating a system-lib.
* This commit implements a CSoureMetaData module to hold
func registry that collects function names from the
runtime module and generates a c source file to be
linked with final artifact.
* Modified and added export_library for utvm
Change-Id: Ie2e8e2aea1a66520f03fe8af7cc5bdf27339ea10
* Created CSourceMetaData module for model metadata
* fixed llvm_module to return null pfs for
get_symbol and get_const_vars
Change-Id: I84810e0695d4d6fb314af2469117f965eed71b51
* Created CSourceMetaData module for model metadata
*fixed bundle_deploy tests
Change-Id: I0d1332a4abbb6830531784c59264021bbbd7148a
* Created CSourceMetaData module for model metadata
*fixed export_library not to insert "options" when targeting tar
*fixed unit tests
Change-Id: Ia1686889498b71af66f1a0311a059154ad3c2c3e
* Created CSourceMetaData module for model metadata
* enable wasm to support csource metadata module
* disabled non DSOExportables from using csource metadata module
Change-Id: Ie09beaad35cbc2ef738d1d24d91e249b5e099569
* Created CSourceMetaData module for model metadata
* changed const pfs to be called only on external modules
or DSOExportable modules
Change-Id: I6ad28f166c0fc27a2548c851bf9287ec805550d1
* Created CSourceMetaData module for model metadata
* CSourceMetadata module wrapper is only created for c/llvm targets
Change-Id: I13cb4140c17e2e1f91d495b15a1ff7eeab9fb14d
* Created CSourceMetaData module for model metadata
*target should be defined to use csourcemetdata module
Change-Id: Id8e55b23d0007a79c550334de2c0fec63d40171f
* Created CSourceMetaData module for model metadata
* reinstate llvm func registry
Change-Id: I53e0754b6fb533637f08b25e98064d8c04092de4
* Created CSourceMetaData module for model metadata
* addressed comments and fixed bugs
Change-Id: I26401685dc803aeaf7642c865df88d683419e859
* Created CSourceMetaData module for model metadata
* addressed a missed comment
Change-Id: I65e65c30bc780a946f3f1b8372c40a49a5c20582
* Created CSourceMetaData module for model metadata
* te build interface should only include c-source metadata if
targetting "c"
Change-Id: Ie23cb8c6231c1f2de6d2827084774e3510288098
* Created CSourceMetaData module for model metadata
* c_source modules should be created only if they are
non-DSO exportable
Change-Id: I53f2f8e9caa41f133446f8881b9dc541ebeee8cc
* Created CSourceMetaData module for model metadata
* documetation misalignment in source_module.cc
Change-Id: I83e2c29b1f2980ca65a694304720dc58a5cb7879
* Created CSourceMetaData module for model metadata
* typo : same object file written as a dependency in the Makefile
Change-Id: I8becc4196d286cfb6372768687b3c836799dcb78
* Created CSourceMetaData module for model metadata
* removed unused param from a brief
Change-Id: Ie4db2aca3b7ea147bd8c65ef5d1cc2146f530e76
* Created CSourceMetaData module for model metadata
* made export library use c as the format for c source modules
Change-Id: Ie2fd6204414f0fa43988a8082d18af7a3225e237
* Created CSourceMetaData module for model metadata
*addressed a nit
Change-Id: I6084b8c06ddfaaece295439dbab589e6e202b664
This PR introduces a minimal set of test cases that
are supposed to run in all platforms during CI.
The set of testcases are supposed to help on
platform dependent regression.
See tests/python/all-platform-minimal-test/README.md for guidelines.
- Enable windows mac LLVM build via conda with cython support.
- Test on all platform test cases.
- Update implementation to improve MSVC support.
* Updated runtime to run under FreeBSD.
setenv CXX to proper binary - c++ or g++9 for FreeBSD 12.0.
* Update python/tvm/runtime/module.py
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
* Update python/tvm/rpc/server.py
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
* Changed to use os.environ.get
* Fixed format.
* Yet another lint fix.
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
* [Relay][VM] Allow to config allocator type and refactor vm code structure
* fix doc
* fix
* update
* trigger ci
* trigger ci
* trigger ci
* trigger ci
* fix doc warning
We use a single enum(TypeCode) to represent ArgTypeCode and DLDataTypeCode.
However, as we start to expand more data types, it is clear that argument
type code(in the FFI convention) and data type code needs to evolve separately.
So that we can add first class for data types without having changing the FFI ABI.
This PR makes the distinction clear and refactored the code to separate the two.
- [PY] Separate ArgTypeCode from DataTypeCode
- [WEB] Separate ArgTypeCode from DataTypeCode
- [JAVA] Separate ArgTypeCode from DataTypeCode
This PR introduces WebGPU support to tvm.
The WebGPU runtime is directly built in javascript(as WebGPU uses JS as the first class citizen API)
and exposes back to the tvm's runtime via PackedFuncs.
One important note is that `ctx.sync` is not async.
This is due to the fact that WebGPU is a purely async API and we cannot block in the web environment.
So the current best way to use the js api is to wrap things in an async function.
When copy a GPU array to CPU, `await ctx.sync()` need to be called to wait for copy completion.
We use a AsyncIO rpc server to serve the async functions to the clients.
* Update dmlc-core which was mistakenly overriden
* [REFACTOR][RPC][PROCOTOL-CHANGE] Modularize the RPC infra.
This PR refactors the RPC protocol to make it more modularized.
- RPCSession: represent a set of features that need to be implemented
- RPCEndPont: End point that forwards the RPCSession requests over a communication channel.
- RPCModule: Exposes an RPCSession as an rpc device in the TVM Runtime API.
In the new design, the local machine is presented as a special case of RPCSession.
The remote is just another client session that calls into RPCEndPoint.
The RPC communication path is as follows.
```
client -> ClientSession -> EndPoint[client@n0]
-> networking[between n0 <=> n1]
-> EndPoint[server@n1] -> LocalSession[@n1]
```
Because of the new modular design, we can now chain more sessions together.
For example, we can now run the following proxy setup (testcase in test_runtime_rpc.test_session_constructor).
```
client -> ClientSession -> Endpoint[client@n0]
-> networking[between n0 <=> n1]
-> Endpoint[server@n1] -> ClientSession -> Endpoint[client@n1]
-> networking[between n1 <=> n2]
-> Endpoint[server@n2] -> LocalSession[@n2]
```
We can also implement other types of Sessions.
As an example, We introduced a PopenSession that communicates with
the another process via a pipe.
We also add more comments about the internal of the RPC.
The communication protocol is simplfied using a similar convention as PackedFunc.
This allows us to further reduce the amount of special remote syscalls.
Due to the major improvement and simplification, we are making a non-compatible update to the RPC protocol.
It means that the client and server needs to be upgraded to together in order for it to function correctly.
This PR also introduces a versioning mechanism to the current RPC procotol,
so that future upgrade will be produce more user friendly with error messages.
* Address review comments
* Remove ld library path
* Prototype for micro TVM.
* Cleanup and sync micro tvm prototype.
* Use /std:c++14 with MSVC.
* Per tqchen: project has already moved to C++14
* Presubmit failed for code that built locally on gcc.
* fix ASF lint, and fix add_asf_header too
* Compiles with USE_MICRO=OFF.
* Cleanup TargetPtr and word size representations.
* fix compile warning
* address logan's comments
* address logan and liangfu comments
* address thierry's comments
* address u99127, liangfu, tmoreau89 comments
Co-authored-by: Logan Weber <weberlo@cs.washington.edu>
The _type_child_slots can be used to enable quick type checking optimization
by checking the whether the type index is within the bound.
This PR enables these static slots:
- Introduce a static assert to avoid the scenario when a developer forget to
_type_child_slots when the field is set for the type's parent.
- Revamp and assign static type index to common runtime objects
- Add a DumpTypeTable call to allow developer monitor the current situation
of type table and offers suggestions for the slots(ideally the slots equals
the number of children so there is no overflow.
To make runtime.String to work as naturally as possible in the python side,
we make it sub-class the python's str object. Note that however, we cannot
sub-class Object at the same time due to python's type layout constraint.
We introduce a PyNativeObject class to handle this kind of object sub-classing
and updated the FFI to handle PyNativeObject classes.
* [RUNTIME] Introduce RValue reference(move) support to TypedPackedFunc
This PR introduces RValue reference support the PackedFunc calling convention to address the above issue.
Specifically, when an argument is a r-value reference, we will use a assign a different type code(`kObjectRValueRefArg`),
and pass `Object**` (the address to the Object pointer) instead through the values array.
The callee can choose to move out this Object pointer and set the original Object pointer from the caller side to be nullptr.
We also add an experimental move support to the python side(marked as _move so to indicate the dev nature).
This enhancement will enable copy on write optimizations through out the TVM stack.
* Address review comments
* fix compilation
* [RUNTIME] Initial implementation of Hexagon runtime support
This is only the TVM runtime. The FastRPC libraries, simulator driver,
etc. will be provided in subsequent commits.
* Fix pylint complaints
* Fix some more pylint complaints
* Add link to the Hexagon SDK website
* Extract VTCM marker into a common variable
* Implement device->device memory copy
* Disable unsigned PDs by default
* Ensure that --hvx_length is present in sim_args if HVX is enabled
* Remove the line about clang from README.md
Apparently things work with libstdc++.
* Mention to set USE_RPC=OFF when building libtvm_runtime.so for Hexagon
* Remember to use codegen_hvx in validate_hvx_length
* Add a line about minimum version of LLVM
* [REFACTOR][TIR] Migrate all low-level passes to the Pass Manager.
This PR migrates the tvm.lower to return IRModule of PrimFuncs
instead of the LoweredFuncs.
* Remove LoweredFunc.
Haichen Shen