(See https://discuss.tvm.apache.org/t/byoc-supporting-cutlass-byoc-with-collage/12796/6 for
context, which in turn is part of Collage (https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md).
This adds a new 'DSO exportable' runtime module representing the contents of a .o file. It
allows external codegen toolchains to yield a result which:
- Like CSource modules, can be conveyed directly to the final export_library compilation
step for linking into the final .so and saved to a know location without risk the
underlying code artifact will be lost.
- Like DSOLibrary modules, are self contained so that no additional compile-time arguments
need be conveyed from the CSource module to the final export_library command line
Since this is the third flavor of 'DSO exportable' module, add a Module::IsDSOExportable.
Since adding the above, can't resist also adding a Module::ImplementsFunction virtual and
calling it from TEComplier to check if an external codegen function actually provided the
implementation it promised.
Note:
- I've left the existing implementation of runtime.load_module alone which
relinks .o files to .so files.
- Though also contained in the .o metadata, I require static libraries to always
carry their list of exported function names.
This is all pretty stop gap pending a good rework of TVM to supoprt the notion of artifacts
and, perhaps, build rules.
* [RUNTIME] Api to get number of runtime threads
Add `tvm::runtime::threading::NumThreads` and `tvm.runtime.num_threads`
as a way to get the number of threads in use by the TVM runtime.
* check if equal to hardware threads or hardware threads/2
* introduce profile_all_alignments option
* add profile_all_alignment option to API
* wip
* fixed dynamic case
* black
* update gen_gemm too
* minor improvement
* fix
* all tests work
* add doc
* fixed for sm = 75 case
* fix typo
* remove unused import
* profile_all -> find_first_valid
* fix
* Allow export of C++ kernels using correct file extension
* [WIP] Set module_key=c for CSourceCrtMetadataModuleNode to temporarily fix failing tests
I realized that the module format `cc` is currently already used by the `CSourceCrtMetadataModuleNode` declared in `src/target/source/source_module.cc`.
This needs to be discussed first to decide if either the module_key should be changed or the test cases expecting the systemlib kernel (e.g. `default_lib0.c`) to have a `.c` extension.
* Update Makefiles used by tests/python/relay/aot/ to support C++ file extensions
AOT: Add c++ support to aot_test.mk
AOT: Add c++ support to corstone300.mk
* Add missing definition of GetFormat to cmsisnn and ethosn codegens (WIP)
* Resolve PR comments
* lint python/tvm/runtime/module.py
* fix EthosUModuleNode for CI
* Fix: detect empty module.format
* Add error message to assertion
* Lint python/tvm/runtime/module.py
* [GRAPH EXECUTOR,VM] Add benchmarking function to graph executor and vm
This new benchmarking function is just a convenience function for
calling time_evaluator on the underlying module. Hopefully this should
make it easier for users to get good benchmarks of their code.
* formatting
* import order
* more test, more comments, more precision
* fix tests
* add seconds descriptions to doc
* [Codegen] Use "target.build.$TARGET_KIND" for all codegen functions.
- Removed special case for "micro_dev" target. Instead, register
BuildCHost as both "target.build.c" and "target.build.micro_dev".
- Renamed "target.build.build.aocl_sw_emu" to
"target.build.aocl_sw_emu". Appears to be a typo introduced in
#841725cc585
* [micro_dev] Removed references to non-existent micro_dev
device_api.micro_dev was removed in
745e542e4d, but several references still
remained.
Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
* Get basic verison of VM RPC working
* Test case passes
* Clean up PR
* Lint
* Format
* Address Andrew R and TK feedback
* Add comment for Andrew
* Address Zhi's comment
* Format
* Fix broken test
* Some docstring fixes.
* Couple of small fixes:
- Use `west attach` instead of `west debug` in commandline to prevent
debugger from resetting device.
- Fix warning on use of led_pin in zephyr-runtime/src/main.c.
* Adding Zephyr demo runtime.
* Cleanup of uTVM tests and demo runtime.
* Working on QEMU support.
Need to add board-specific prj.conf files.
* Adding board-specific prj.conf files.
* Some cleanup.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Adding data for unit tests.
* Cleanup demo_runtime code.
* Fix up tutorial.
* Couple of small fixes:
- Use `west attach` instead of `west debug` in commandline to prevent
debugger from resetting device.
- Fix warning on use of led_pin in zephyr-runtime/src/main.c.
* Adding Zephyr demo runtime.
* Cleanup of uTVM tests and demo runtime.
* Working on QEMU support.
Need to add board-specific prj.conf files.
* Adding board-specific prj.conf files.
* Some cleanup.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Lots of hacking to get ONNX model to run on QEMU and nRF5340.
Added test_onnx unit test.
Still need to clean up tutorial.
* Adding data for unit tests.
* Cleanup demo_runtime code.
* Fix up tutorial.
* Fix tutorial.
* Fix tutorial and runtime.
* Fix merge conflicts.
* Fix merge conflict.
* Remove redundant files.
* Revert dep.
* Fixup
* Add new files to check_file_type.py.
* Adding missing ONNX file.
* Fixup docs.
* Fix linting rule.
* small fixes
* Add missing file to check_file_type.py.
* clang-format this file.
* Fix formatting.
* Black formatting.
* Lint comments.
* Fix path for test.
* Bump CI.
* Update from_onnx.
* fix path
* Fixing
* Revert dmlc-core to 21cc7de0dc9fd6acb796e1be6181fa8e6b6c8f41
* Fix path again.
* Fix tutorial to not use actual Zephyr.
* Revert submodule version change
* Fix bad merge.
* Trying to fix this mess.
* Fix formatting.
* context -> device
* Removing tutorial since I can't get it to pass CI.
Co-authored-by: Mehrdad Hessar <mehrdad.hessar@gmail.com>
Co-authored-by: Andrew Reusch <areusch@octoml.ai>
* Introduce Model Library Format export format.
* This function produces a stable on-disk representation of TVM's
compiler output.
* It's intended just for use with the C runtime for microTVM right
now. It could be expanded for other use cases.
* This PR implements the Model Library Format RFC, which ultimately
is intended to support the Project Generator API (RFC
forthcoming).
* There may be some changes to the format without revving the version
number until downstream consumers are known. The Project Generator
API is the first such known downstream consumer.
* There are no plans currently to support generating old Model
Library Format from TVM. The version number is intended as a
compatibility check between the generator and downstream consumers.
* Created CSourceMetaData module for model metadata
* Currently, there is a MetaData module to capture constants
conditionaly if the runtime modules implement const init
PackedFuncs. However, this one relies on a load process
in which the metadata is created on volatile memory that
may be not usable in uTVM environments.
* There is a need for model level metadata that is valid
across all runtime modules such as the func registry
when creating a system-lib.
* This commit implements a CSoureMetaData module to hold
func registry that collects function names from the
runtime module and generates a c source file to be
linked with final artifact.
* Modified and added export_library for utvm
Change-Id: Ie2e8e2aea1a66520f03fe8af7cc5bdf27339ea10
* Created CSourceMetaData module for model metadata
* fixed llvm_module to return null pfs for
get_symbol and get_const_vars
Change-Id: I84810e0695d4d6fb314af2469117f965eed71b51
* Created CSourceMetaData module for model metadata
*fixed bundle_deploy tests
Change-Id: I0d1332a4abbb6830531784c59264021bbbd7148a
* Created CSourceMetaData module for model metadata
*fixed export_library not to insert "options" when targeting tar
*fixed unit tests
Change-Id: Ia1686889498b71af66f1a0311a059154ad3c2c3e
* Created CSourceMetaData module for model metadata
* enable wasm to support csource metadata module
* disabled non DSOExportables from using csource metadata module
Change-Id: Ie09beaad35cbc2ef738d1d24d91e249b5e099569
* Created CSourceMetaData module for model metadata
* changed const pfs to be called only on external modules
or DSOExportable modules
Change-Id: I6ad28f166c0fc27a2548c851bf9287ec805550d1
* Created CSourceMetaData module for model metadata
* CSourceMetadata module wrapper is only created for c/llvm targets
Change-Id: I13cb4140c17e2e1f91d495b15a1ff7eeab9fb14d
* Created CSourceMetaData module for model metadata
*target should be defined to use csourcemetdata module
Change-Id: Id8e55b23d0007a79c550334de2c0fec63d40171f
* Created CSourceMetaData module for model metadata
* reinstate llvm func registry
Change-Id: I53e0754b6fb533637f08b25e98064d8c04092de4
* Created CSourceMetaData module for model metadata
* addressed comments and fixed bugs
Change-Id: I26401685dc803aeaf7642c865df88d683419e859
* Created CSourceMetaData module for model metadata
* addressed a missed comment
Change-Id: I65e65c30bc780a946f3f1b8372c40a49a5c20582
* Created CSourceMetaData module for model metadata
* te build interface should only include c-source metadata if
targetting "c"
Change-Id: Ie23cb8c6231c1f2de6d2827084774e3510288098
* Created CSourceMetaData module for model metadata
* c_source modules should be created only if they are
non-DSO exportable
Change-Id: I53f2f8e9caa41f133446f8881b9dc541ebeee8cc
* Created CSourceMetaData module for model metadata
* documetation misalignment in source_module.cc
Change-Id: I83e2c29b1f2980ca65a694304720dc58a5cb7879
* Created CSourceMetaData module for model metadata
* typo : same object file written as a dependency in the Makefile
Change-Id: I8becc4196d286cfb6372768687b3c836799dcb78
* Created CSourceMetaData module for model metadata
* removed unused param from a brief
Change-Id: Ie4db2aca3b7ea147bd8c65ef5d1cc2146f530e76
* Created CSourceMetaData module for model metadata
* made export library use c as the format for c source modules
Change-Id: Ie2fd6204414f0fa43988a8082d18af7a3225e237
* Created CSourceMetaData module for model metadata
*addressed a nit
Change-Id: I6084b8c06ddfaaece295439dbab589e6e202b664
* Updated runtime to run under FreeBSD.
setenv CXX to proper binary - c++ or g++9 for FreeBSD 12.0.
* Update python/tvm/runtime/module.py
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
* Update python/tvm/rpc/server.py
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
* Changed to use os.environ.get
* Fixed format.
* Yet another lint fix.
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
This PR introduces WebGPU support to tvm.
The WebGPU runtime is directly built in javascript(as WebGPU uses JS as the first class citizen API)
and exposes back to the tvm's runtime via PackedFuncs.
One important note is that `ctx.sync` is not async.
This is due to the fact that WebGPU is a purely async API and we cannot block in the web environment.
So the current best way to use the js api is to wrap things in an async function.
When copy a GPU array to CPU, `await ctx.sync()` need to be called to wait for copy completion.
We use a AsyncIO rpc server to serve the async functions to the clients.
* Update dmlc-core which was mistakenly overriden
* [REFACTOR][RPC][PROCOTOL-CHANGE] Modularize the RPC infra.
This PR refactors the RPC protocol to make it more modularized.
- RPCSession: represent a set of features that need to be implemented
- RPCEndPont: End point that forwards the RPCSession requests over a communication channel.
- RPCModule: Exposes an RPCSession as an rpc device in the TVM Runtime API.
In the new design, the local machine is presented as a special case of RPCSession.
The remote is just another client session that calls into RPCEndPoint.
The RPC communication path is as follows.
```
client -> ClientSession -> EndPoint[client@n0]
-> networking[between n0 <=> n1]
-> EndPoint[server@n1] -> LocalSession[@n1]
```
Because of the new modular design, we can now chain more sessions together.
For example, we can now run the following proxy setup (testcase in test_runtime_rpc.test_session_constructor).
```
client -> ClientSession -> Endpoint[client@n0]
-> networking[between n0 <=> n1]
-> Endpoint[server@n1] -> ClientSession -> Endpoint[client@n1]
-> networking[between n1 <=> n2]
-> Endpoint[server@n2] -> LocalSession[@n2]
```
We can also implement other types of Sessions.
As an example, We introduced a PopenSession that communicates with
the another process via a pipe.
We also add more comments about the internal of the RPC.
The communication protocol is simplfied using a similar convention as PackedFunc.
This allows us to further reduce the amount of special remote syscalls.
Due to the major improvement and simplification, we are making a non-compatible update to the RPC protocol.
It means that the client and server needs to be upgraded to together in order for it to function correctly.
This PR also introduces a versioning mechanism to the current RPC procotol,
so that future upgrade will be produce more user friendly with error messages.
* Address review comments
* Remove ld library path
* Prototype for micro TVM.
* Cleanup and sync micro tvm prototype.
* Use /std:c++14 with MSVC.
* Per tqchen: project has already moved to C++14
* Presubmit failed for code that built locally on gcc.
* fix ASF lint, and fix add_asf_header too
* Compiles with USE_MICRO=OFF.
* Cleanup TargetPtr and word size representations.
* fix compile warning
* address logan's comments
* address logan and liangfu comments
* address thierry's comments
* address u99127, liangfu, tmoreau89 comments
Co-authored-by: Logan Weber <weberlo@cs.washington.edu>
* [REFACTOR][PY-API] Polish tvm.runtime, tvm.runtime.module API update
This PR updates the tvm.runtime to use the new FFI style.
- Remove top-level tvm.module to avoid confusion between runtime.Module and IRModule
- API changes wrt to runtime.Module
- tvm.module.load -> tvm.runtime.load_module
- tvm.module.enabled -> tvm.runtime.enabled
- tvm.module.system_lib -> tvm.runtime.system_lib
- Remove dep on api_internal from runtime.
* Update module.load in the latest API
* [REFACTOR][PY] Establish tvm.runtime
This PR establishes the tvm.runtime namespace that contains the core runtime data structures.
The top-level API are kept inact for now via re-exporting.
We will followup later to cleanup some of the top-level APIs.
* Fix ndarray name
Mark Shields