* Finish support for list-of-targets
This finishes the work started in https://github.com/apache/tvm/pull/11173 to support
'external codegen' targets in the N build-like API surfaces.
- It turns out it's ok if a build is given only a single 'external codegen' target, so remove that check
in CompilationConfig::Init. When Collage builds a 'candidate partition' it does so for a single target.
As far as Collage is concerned it does not care whether the target is regular (eg Target("cuda")), or
for a specific external codegen (eg Target("cutlass")), it just passes the target into the build.
- Add CompilationConfig::FindPrimitiveTargetForKind which I'll later need to retrieve
the external codegen Target instance corresponding to a "Compiler" attribute value.
- Target.update_target_host_consist was supporting three API styles:
- single target
- map from device type to target
- map from target to IRModule (for the ir_to_runtime API)
I replaced all those calls with a more specialized 'canonicalize' call:
- Target.canonicalize_target_and_host
- Target.canonicalize_multi_targets_and_host
- Target.canonicalize_target_map_and_host
In particular, all the tuning interfaces (task extraction, tuning, tuning records) all explicitly
*do not* support multiple targets since the underlying code just doesn't support that.
* - Lints
- Revert unintended changes
* - more lints
* - Fix model_library_format handling of target.
- Improve comments in compilation_config.h
* - Lints
- Update target/target_host params documentation
* - Fix micro library format tests
- Rev micro library format from 5 to 6
- Use Target.current() in a few places
* - eta contract comprehension
* - Woops, one more device: target map left
- Handle host already being in Target
* - lint
* - lint
* - Bug with append
- Take device type from target
* - Fix hexagon
* fix bugs in the auto scheduler record:
* reformat the code
* reformat the code
* use the os.path.abspath
* change error to warning
* reformat the warning code
* fix some typos
* fix some typos
* fix some typos
* fix the port number typo
* Rename references to 0.0.0.0 to localhost. Also change references to 127.0.0.1 to localhost so that all references are consistent. 0.0.0.0 is not the same as localhost.
Previously the rpc server relies multiprocessing to start a new process and does not work under jupyter.
It also have a popen mode that does ensure the socket start listening before returning the port number.
This PR switches the implementations use PopenWorker. The port number is returned after the socket
get binded, which resolves some of the RPC flaky issues(need sleep to wait the server to start).
It also makes the RPC server jupyter friendly.
* Fix legacy code on target host
* Modify legacy code for target host change
* Add tests and fix merge issue
* Add condition for same host
* Modify all files for new target host api compatibility
* Add newline
* Change import format
* Optimize test file
* Add match error info for unit tests
* Fix for heterogeneous targets
* Fix format for dict iteration
* Fix target host type error
* Skip one testcase for tvm infinite loop bug
* Fixed bug for target map compatibility
* Fix another TargetsMap issue
* Fix typo and infinite loop error
* Temporary fix for handle issue
* Fix vm target
* Add condition support for str case
* Add GetHost function and fix previous bugs
* Fix measure_record.cc
* Fix search_task.cc
* Fix compiler.cc, memory_alloc.cc
* Fix driver_api.cc
* Fix format
* Fix bugs and GetHost function usage
* Fix clang format
* Fix bug
* Modify python tests
* Change python unit tests to new target api
* Fi test_runtime_heterogeneous.py
* Modify tutorials & remove extra print
* Update more tests to new api
* Refine the tutorial target usage
* change argument name for Target constructor function
* Fix target export function
* Fix and validate all tutorial usage
* Remove unused argument
* Fix format
* Fix bug in driver/build_module.py for heterogeneous target
* Fix bug in driver/build_module.py for heterogeneous target more
* Fix target host type error
* Fix cudnn target host bug
* Fix according to reviews, add helper function in python
* Refactor code as helper function
* Expand helper function
* Fix bug add and update python helper function
* Update target hosts
* Fix format & refresh function
* Fix unit test bug
* Fix bug in refreshing host
* Fix bug
* Add SetHost function
* Update export function
* Fix format
* Fix export bug in target
* Fix bug on host referencing
* Addtional tests
* Address review issues
* Fix format target.py
* Fix issues and format
* Add some 3rd party dependencies
* Merge main branch
* Fix target.h format
* Remove redundent import
* Fix function name
* Add parameter name
* Fix new code bug
* Fix bug in lowering
PopenPoolExecutor implements a ProcessPoolExecutor backed by popen.
- Only handles invoking functions in tvm namespace.
- Unlike multiprocessing, does not require __main__ block,
which means it can directly run on jupyter notebook.
- Come with timeout and fault tolerant support to timeout
long running jobs, and restart the process when an error happens.
Recommended usage: it is recommended to create a pool and reuse
it in a long running job(e.g. autotuning) so that the process
are reused when possible.
* [BUGFIX] Respect infinite-timed session start timeouts.
* When debugging, the intended behavior is to set the session start
timeout to infinite to allow the user to configure the debugger.
* At present, if a session start retry timeout is defined, the
current logic will bail after the retry timeout expires.
* This change makes the session start logic retry forever, once per
retry timeout.
* Document RPCEndpoint::Create.
* Add stm32f746xx to tvm.target.micro() call; fix parameter name.
* This API is expected to just be used with positional args, not
kwargs, so this change isn't expected to cause any breakage.
* model is more inline with the rest of the file, given TVM Target
Specification RFC.
* [BUGFIX] If session start fails, exit transport context manager.
* If an error occurred during session setup, then complex transports
e.g. DebugWrapperTransport would not de-initialize.
* Align transport writes/reads in TransportLogger
* fix syntax errors which were not exercised in previous PR
* Remove microTVM logic from standard RPC server, add debug shell.
* microTVM uses the host RPC server as a way to launch a debugger in
a dedicated, separate terminal window. microTVM needs to be able to
launch the debugger itself, because its model of the device
flash/debug flow separates these two things into distinct
operations implemented by shell commands (for maximum portability
across frameworks).
* microTVM can be configured to launch the debugger (e.g. GDB) in the
same terminal as is used for flashing, but this is sub-optimal
because then it hides any logs emitted by the device.
* Using the standard RPC server was hard because GDB expects the user
to issue SIGINT to interrupt program flow, but due to the RPC
server's necessary use of multiprocessing, multiple signal handlers
needed to be SIG_IGN'd, and further, because libtvm.so is
intentionally frontend-agnostic, it's difficult to include signal
handling directly in that binary (Python expects you to call
PyErr_CheckSignals, but we don't require and don't want to require
python-dev to compile libtvm.so, and this is the only such case
where libtvm.so is expected to block the main thread for a long
period of time).
* Here we implement a separate microTVM debug shell python script
using the non-blocking server implementation.
* Add serial transport, parameterize test_zephyr to work on real hardware
* add pytest test fixture, missed from previous change.
* this test fixture helps to parameterize the test case
* address leandron@ comment from #6703
* Split transport classes into transport package.
* Introduce transport timeouts.
* black format
* Add metadata-only artifacts
* Simplify utvm rpc server API and ease handling of short packets.
* add zephyr test against qemu
* Add qemu build config
* fix typo
* cleanup zephyr main
* fix nonblocking piping on some linux kernels
* don't double-open transport
* validate FD are in non-blocking mode
* gitignore test debug files
* cleanup zephyr compiler
* re-comment serial until added
* remove logging
* add zephyr exclusions to check_file_type
* add asf header
* lint
* black format
* more pylint
* kill utvm rpc_server bindings, which don't work anymore and fail pylint
* fix compiler warning
* fixes related to pylint
* clang-format again
* more black format
* add qemu regression
* Fix paths for qemu/ dir
* fix typo
* fix SETFL logic
* export SessionTerminatedError and update except after moving
* fix test_micro_artifact
* retrigger staging CI
* fix jenkins syntax hopefully
* one last syntax error
* Add ci_qemu to Jenkinsfile
* build in qemu
* address liangfu comments
* fix new bug with list passing
* retrigger CI
* [RPC] Lazily import micro when starting an RPC server
Since #6334 the RPC server cannot be started unless USE_MICRO is enabled. I've tracked this down to an import in `python/tvn/exec/rpc_server.py`: `from tvm import micro` in the top level list of imports. This will mean that we try to import micro when it's not been built. Fix this by lazily importing micro when initializing an rpc server with micro enabled.
Change-Id: I8f22d81e215cfe4ac0662b0a99bdf02a3e91f90c
* fix lint
Change-Id: I8b78b678374bc82b3b66a7b3595ed4f1684e7d90
This PR introduces WebGPU support to tvm.
The WebGPU runtime is directly built in javascript(as WebGPU uses JS as the first class citizen API)
and exposes back to the tvm's runtime via PackedFuncs.
One important note is that `ctx.sync` is not async.
This is due to the fact that WebGPU is a purely async API and we cannot block in the web environment.
So the current best way to use the js api is to wrap things in an async function.
When copy a GPU array to CPU, `await ctx.sync()` need to be called to wait for copy completion.
We use a AsyncIO rpc server to serve the async functions to the clients.
* [WEB] Remove the old web runtime
* [WEB][RUNTIME] TVM WebAssembly Runtime
This PR introduces a brand new TVM web runtime based on the WASM standard API.
Main highlights:
- The new runtime is rewritten using the Typescript.
- The new runtime now directly interfaces with WebAssembly's standard API,
instead of relying on emscripten's API.
This change will make the js runtime more portable to runtime variants.
For example, we could also try to make it interface with the tvm's rust runtime implementation.
- System library can be provided through WASI
- We also build a hack to enable Emscripten to generate a WASI like
bundle for runtime environment on the Web.
- The wasm generation now uses the mainlin LLVM.
- Dynamic link(dlopen) is not used due to limitation of wasm,
instead we rely on the recent new RPC refactor to directly
restart a new session for each wasm binary sent to the RPC.
* Address review comments
* Skip tensorcore test
* Prototype for micro TVM.
* Cleanup and sync micro tvm prototype.
* Use /std:c++14 with MSVC.
* Per tqchen: project has already moved to C++14
* Presubmit failed for code that built locally on gcc.
* fix ASF lint, and fix add_asf_header too
* Compiles with USE_MICRO=OFF.
* Cleanup TargetPtr and word size representations.
* fix compile warning
* address logan's comments
* address logan and liangfu comments
* address thierry's comments
* address u99127, liangfu, tmoreau89 comments
Co-authored-by: Logan Weber <weberlo@cs.washington.edu>
Mark Shields