107 Commits

Author SHA1 Message Date
Tianqi Chen 96cba60464 [PYTHON] Autoload backends; simplify library loading; remove TVMError for native errors (#19727)
This PR adds an autoload mechanism for out-of-tree backends, simplifies
TVM's Python library loading, and removes `TVMError` in favor of native
Python errors.

## Autoload out-of-tree backends

Out-of-tree packages can register an autoload callable under the
`tvm.backends` entry-point group (mirroring torch's device-backend
autoload). At `import tvm` startup each entry point is discovered and
its callable invoked once, after the core runtime and the `tvm`
namespace are fully initialized, so an extension can register
ops/targets/funcs or load extra libraries.

```toml
[project.entry-points."tvm.backends"]
tvm_foo = "tvm_foo:_autoload"
```

A failing extension is caught and surfaced via `warnings.warn` so it
cannot break `import tvm`. Autoload can be disabled with
`TVM_DEVICE_BACKEND_AUTOLOAD=0`.

## Simplify library loading

The library-loading path in `base.py` is consolidated around a single
`_LOADED_LIBS` dict (basename to ctypes handle) so downstream and
autoloaded extensions can skip already-loaded libraries; the per-backend
runtime DSO list is folded into `load_backend_libs`. Accumulated cruft
is removed: the Python-3.9 check, the readline shim, the `_FFI_MODE`
ctypes check, the `base.__version__` re-export, and `py_str` (call sites
inline `.decode("utf-8")`).

## Remove TVMError in favor of native Python errors

`TVMError` added a layer atop `RuntimeError` that downstream code had to
import and learn. It is removed; the registered FFI error kinds
(`InternalError`, `RPCError`, `OpError`, `DiagnosticError`,
`ScheduleError`) now subclass `RuntimeError` directly while staying
registered, so the FFI keeps throwing the right kinds. All `TVMError`
imports, `except`/`raise`/`isinstance` uses, and
`pytest.raises(tvm.TVMError)` sites move to the `RuntimeError` builtin.
2026-06-11 13:50:38 -04:00
Tianqi Chen ebc9348bd7 [REFACTOR][PYTHON] Slim tvm.libinfo to info-only helpers (#19719)
This PR slims `tvm.libinfo` into a thin *info* layer that delegates path
discovery to the `tvm_ffi.libinfo` primitives and never loads libraries.
Loading responsibilities move to `tvm.base`, and the various ad-hoc
path-finding helpers are phased out in favor of the tvm-ffi resolvers.

## Changes

- **libinfo**: add `find_libtvm_runtime()` (resolves `libtvm_runtime`
via
  `_find_library_by_basename` + `_resolve_and_validate`) and
`find_tvm_include_path()` (TVM's own `include/`). `find_include_path()`
now
returns `[find_tvm_include_path(), *tvm_ffi.libinfo.include_paths()]`,
folding
in the FFI + dlpack + python-helper include dirs. Remove
`find_lib_path`,
  `get_dll_directories`, `use_runtime_lib`, `split_env_var`, and
  `load_backend_libs`.
- **base**: receive `load_backend_libs` and the backend DSO list; the
runtime-only switch becomes a strict `TVM_USE_RUNTIME_LIB == "1"` check.
- **rpc**: `with_minrpc` uses `find_libtvm_runtime()` (the `runtime`
kwarg is
retained as an inert back-compat parameter); the rpc server
`load_library`
resolves the literal library name against the current working directory.
- **wasm**: move the `web/dist` asset search into `emcc.find_wasm_lib`,
used by
  `emcc.create_tvmjs_wasm` and the tvmjs asset lookup.
- **hexagon**: fix a latent bug where `_get_hexagon_rpc_lib_dir` called
a
non-existent `tvm_ffi.libinfo.find_lib_path`; it now relies solely on
the
  `HEXAGON_RPC_LIB_DIR` environment variable.
2026-06-10 13:41:37 -04:00
Shushi Hong a979b2f98c [RPC] Import tvm.testing lazily in rpc.testing (#19658)
### Motivation

`tvm.testing` imports `pytest` at module load (`tvm/testing/utils.py`).
`tvm.rpc.server` imports `tvm.rpc.testing` (to register the `rpc.test.*`
helpers), and `tvm.rpc.testing` imported `tvm.testing` at the top level,
so a plain `import tvm` / `import tvm.relax` pulls `pytest` in through:

```
tvm.relax -> tvm.runtime.vm -> tvm.rpc -> rpc.server -> rpc.testing -> tvm.testing -> pytest
```

As a result `pytest` is effectively a runtime dependency: a user who
installs TVM without `pytest` hits `ModuleNotFoundError: No module named
'pytest'` on import. This is easy to miss because test environments
install `pytest`.

### Change

`tvm.rpc.testing` only uses `tvm.testing.object_use_count` in a single
test helper, so import it lazily at the call site instead of at module
top level. This keeps the `rpc.test.*` registration and the helper
behavior intact while removing `tvm.testing` (and `pytest`) from the
`import tvm` path, so `pytest` can remain a test-only dependency.

No functional change; `rpc.testing` is still imported by `rpc.server`
and still registers the same global functions.
2026-06-02 18:22:38 -04:00
Tianqi Chen 23db01f1fa [REFACTOR][RUNTIME] Structural reorganization: locality moves for thread_map, texture, minrpc, disco, contrib (#19628)
## Background

The TVM runtime has been growing organically. Several headers and
directories
live at the top level of `src/runtime/` despite only being consumed by a
single backend subsystem. This PR applies the **locality principle**:
code that
has exactly one consumer moves to live next to that consumer.

## Changes

### Move 1: `thread_map.h` → `src/runtime/vulkan/`
`ThreadMap` is only used by Vulkan device API headers. Moving it under
`src/runtime/vulkan/` reflects this exclusive ownership.

### Move 2: `texture.h` → `src/runtime/opencl/`
Texture storage utilities are OpenCL/Adreno-specific. Moving the header
under `src/runtime/opencl/` makes ownership clear.

### Move 3: `minrpc/` → `src/runtime/rpc/minrpc/`
The minrpc mini-RPC implementation belongs logically under the existing
`src/runtime/rpc/` subtree. All consumers already live under rpc/ or
reference it as a child of rpc/.

### Move 4: Introduce `src/runtime/extra/` boundary
`disco/` and `contrib/` are the sole source directories for
`libtvm_runtime_extra`. Grouping them under `src/runtime/extra/` makes
the
`libtvm_runtime_extra` build boundary visible in the filesystem,
matching
the modular runtime split introduced in #19444.
- `src/runtime/disco/` → `src/runtime/extra/disco/`
- `src/runtime/contrib/` → `src/runtime/extra/contrib/`
- Public `include/tvm/runtime/disco/` is unchanged.

### Drive-by fixes
- `apps/android_rpc/…/tvm_runtime.h`: Drop stale `minrpc_logger.cc`
include
(file no longer exists) and fix stale `tvm-ffi/src/ffi/extra/testing.cc`
  path to `tvm-ffi/src/ffi/testing/testing.cc`.

## Test Plan

- [x] Full build (`ninja -j$(nproc)`) — succeeds
- [x] `./cpptest` — 118 tests passed
- [x] Python smoke: `tvm.__version__` + `tvm.cuda(0).exist` — pass
- [x] `tests/python/all-platform-minimal-test` — 37 passed, 105 skipped
- [x] `tests/python/runtime/test_runtime_rpc.py` — 2 passed, 21 skipped
- [x] `tests/python/runtime/test_rpc_base.py` — 2 passed
- [x] `pre-commit run --all-files` — all hooks pass
2026-05-27 17:42:43 -04:00
Tianqi Chen ffea531107 [REFACTOR][PYTHON] Lift compiler/CLI/process modules from tvm.contrib to tvm.support (#19624)
## Summary

Lifts 10 host-toolchain / CLI / process / utility modules from
`python/tvm/contrib/` to a new `python/tvm/support/` package, and
deletes two dead contrib shims.

`tvm.support` is the home for Python helpers that integrate TVM with
external CLIs and host-side tools — compilers, archivers, subprocess
pools, and build-info queries. These are load-bearing internal pieces
that TVM's compile/link/run paths depend on. `tvm.contrib` is reserved
for optional vendor SDK integrations and experimental features. The
distinction is documented in the `tvm.support` package docstring.

Moved (one commit each):

- `tvm.contrib.cc` → `tvm.support.cc`
- `tvm.contrib.nvcc` → `tvm.support.nvcc`
- `tvm.contrib.rocm` → `tvm.support.rocm`
- `tvm.contrib.ndk` → `tvm.support.ndk`
- `tvm.contrib.xcode` → `tvm.support.xcode`
- `tvm.contrib.clang` → `tvm.support.clang`
- `tvm.contrib.emcc` → `tvm.support.emcc`
- `tvm.contrib.popen_pool` → `tvm.support.popen_pool`
- `tvm.contrib.utils` → `tvm.support.utils`
- `tvm.contrib.tar` → `tvm.support.tar`

Deleted:
- `tvm.contrib.spirv` — single `optimize()` wrapping `spirv-opt`; zero
importers.
- `tvm.contrib.rpc` — self-deprecation shim with "removed in 0.5"
banner; honoring it.

Package conversion:
- `python/tvm/support.py` → `python/tvm/support/__init__.py` with
inclusion-rule docstring.
- `libinfo()` extracted into `python/tvm/support/libinfo.py`.
- `FrontendTestModule` dropped (audit confirmed zero callers outside its
own definition).

## Compatibility

Hard break — no `tvm.contrib.<mod>` re-export shims. All callers updated
in this PR.

C++-side FFI registry keys (`tvm.contrib.nvcc.*`, etc.) are unchanged —
only the Python module path moves. Renaming the FFI keys is a separate
follow-up.
2026-05-27 15:31:12 -04:00
Shushi Hong cae6cb89b7 [IR] Add annotations to Call nodes (#19597)
This PR adds annotation support to `tirx.Call` so downstream codegen
users can attach call-level metadata and preserve it through TIRX
transforms.

What changed:
- Add `CallNode::annotations` and expose it through reflection.
- Add Python `tvm.tirx.Call(..., annotations=...)` support.
- Preserve call annotations in C++ and Python expression mutators.
- Preserve annotations across TIRX/arith passes that rebuild equivalent
calls.
- Print annotated calls as `Tx.Call(..., annotations={...})` and support
script roundtrip.
- Add regression coverage for annotated calls, mutator preservation,
script roundtrip, and simplify preservation.

This pr also cleans some stuff that #19596 didn't clean completely
2026-05-24 18:57:37 -04:00
Bl4ckSku11 a7463e9b2d [RPC][Tracker] Bound msg_size to MAX_TRACKER_MSG_BYTES to prevent unbounded buffer growth (#19586)
Fixes #<issue-number>.

Reads of `_msg_size` from the tracker socket are now bounded to
`MAX_TRACKER_MSG_BYTES = 1 MiB`, and the 4-byte size header is
consumed at read time. Without these checks, a single TCP connection
from a peer can grow the tracker process buffer until OOM, and a wire
size of 0 starves the parser without ever freeing the bytes.

Per the TVM security model the tracker is deployed on trusted networks,
so this is filed as a robustness defect, not a security advisory.
Apache security team triage (private thread, 2026-05-17) confirmed this
is the right channel.

### Test
Added regression test in tests/python/contrib/test_rpc_tracker.py that
completes the magic handshake, sends an oversized msg_size header
(0x7FFFFFFF), and asserts the tracker closes the connection.

### Changes
- python/tvm/rpc/tracker.py: bound `_msg_size` to (0,
MAX_TRACKER_MSG_BYTES], consume size header on read.
- tests/python/contrib/test_rpc_tracker.py: regression test.
2026-05-24 00:07:03 -04:00
Tianqi Chen 8cbb0b11bb [CMAKE][REFACTOR] Split libtvm.so into libtvm_runtime.so and libtvm_compiler.so (#19444)
## Motivation

Historically TVM ships a single monolithic `libtvm.so` that bundles both
the
runtime and the compiler/LLVM-heavy code paths. Deployment scenarios
that only
need the runtime end up paying the full compiler footprint (LLVM-static
dominates
the binary size), and the layout makes it awkward to install the project
under a
single Python package directory the way
`tvm_ffi`/`libinfo.load_lib_ctypes`
expects.

This PR splits the single shared library into two:

- `libtvm_runtime.so` — runtime-only symbols (loaded `RTLD_GLOBAL`).
- `libtvm_compiler.so` — compiler / LLVM / codegen, links
`libtvm_runtime.so`
  publicly (loaded `RTLD_LOCAL`).

## Target restructure

- New CMake target `tvm_compiler` replaces the old `tvm` SHARED target.
- `tvm_compiler` depends on `tvm_runtime` via `target_link_libraries(...
PUBLIC tvm_runtime)`,
so anything that linked the old `tvm` now picks up the runtime
transitively.
- `tvm_libinfo_objs` (build-info TU) moved from `tvm_runtime` into
`tvm_compiler`
  — it is compiler-side metadata and the runtime no longer needs it.
- All `target_link_libraries` / `target_compile_*` /
`set_target_properties` /
  `tvm_ffi_add_apple_dsymutil` callsites have been rewired.
- The separate `libtvm_allvisible.so` target is **removed** (was only
consumed
  by cpptests). Cpptests with private-symbol deps are deleted; remaining
  cpptests now link directly against `libtvm_compiler.so` /
`libtvm_runtime.so`. `src/support/hexdump.cc` is folded into the header.
- `BUILD_DUMMY_LIBTVM` and the `BUILD_FOR_HEXAGON + USE_HEXAGON_GTEST`
  cpp-test wiring are removed.

## Output and install layout

- All artifacts now go to `build/lib/` (was `build/`):
  - `build/lib/libtvm_runtime.so`
  - `build/lib/libtvm_compiler.so`
- Install layout is now `<package>/lib/` so
`tvm_ffi.libinfo.load_lib_ctypes`
  with `package="tvm"` finds the libs in the wheel.
- CI Jenkins stash paths and `apps/hexagon_*` paths updated to the new
  `build/lib/...` location.

## Python loader change

`python/tvm/base.py` now resolves the libs directly via a small
`package_lib_paths()` helper in `python/tvm/libinfo.py` (anchored on
`python/tvm/__file__`, returning the wheel `lib/`,
`<worktree>/build/lib`, and
`<worktree>/lib` candidates). Module-level `_LIB_RUNTIME`, `_LIB`, and
`_RUNTIME_ONLY` are set inline at import time:

- `libtvm_runtime.{so,dylib,dll}` loaded `RTLD_GLOBAL`.
- `libtvm_compiler.{so,dylib,dll}` loaded `RTLD_LOCAL`.
- `TVM_USE_RUNTIME_LIB` (parsed strictly: `1`/`true`/`yes`) selects
  runtime-only at the loader level.
- When the compiler lib is absent, `_RUNTIME_ONLY` is set to True
  automatically and `_LIB is _LIB_RUNTIME`.

## Non-obvious build-integration fixes

Three issues surfaced once both libs are loaded into the same process
and are
worth calling out:

1. **`fpA_intB_gemm` double-registration.** `fpA_intB_gemm_tvm` is an
OBJECT
library that registers a global `fastertransformer.gemm_fp16_int` at
static
   init. Linking it into both `tvm_runtime` and `tvm_compiler` made the
registration run twice and trip the duplicate-registration check. Fix:
link
it (and the other runtime-only externals — `flash_attn`, NCCL, NVSHMEM,
RCCL) only into `tvm_runtime`. `tvm_compiler` picks them up via the
PUBLIC
   `tvm_runtime` link.

2. **`-Wl,--no-as-needed` for minrpc.** `python/tvm/rpc/minrpc.py`
defaults
   to `runtime="libtvm_runtime"` and passes `-Wl,--no-as-needed` so the
   runtime static initializers actually run in the spawned minrpc binary
   (without it, the linker drops the lib because no symbol is referenced
   directly from the minrpc TU). minrpc does **not** link
   `libtvm_compiler.so`.

3. **`testing.GetShape{Elem,Size}` moved to runtime.** Those two test
helpers
(the only `testing.*` symbols the minrpc test exercises) were registered
in
`src/support/ffi_testing.cc` (compiler-side). They are now registered in
`src/runtime/rpc/testing.cc` under `rpc.testing.GetShape{Elem,Size}` so
   the minrpc server binary — runtime-only — can resolve them.

## Deprecations and breaking changes

- `BUILD_DUMMY_LIBTVM` is **removed** (option, libinfo entry, and CMake
  wiring). Downstream consumers that built the dummy variant should link
  `libtvm_runtime.so` directly.
- **Breaking change for downstream consumers** that read `libtvm.so` by
name:
there is no longer a `libtvm.so`. Replace with `libtvm_compiler.so`
(full)
or `libtvm_runtime.so` (runtime-only). The Vulkan device comment and a
few
  test/CI comments have been updated accordingly.
- `libtvm_allvisible.so` is **removed**. Cpptests that depended on
private
out-of-line symbols have been deleted; the remaining cpp-test contract
is
  documented as "public API or private header-only API only" (see
  `tests/cpp/`).
- `tests/cpp-runtime/` (Hexagon + OpenCL backend tests) is **removed**
until
  TVM moves to a plugin-mode backend architecture where each backend can
  ship its own test harness with its own visibility scope.

## Tested

- `ninja` build: `build/lib/libtvm_runtime.so`,
`build/lib/libtvm_compiler.so`;
  no `build/libtvm.so`, no `build/lib/libtvm_allvisible.so`.
  `ldd build/lib/libtvm_compiler.so` links `libtvm_runtime.so`,
  `libtvm_ffi.so`, `libfpA_intB_gemm.so`, `libflash_attn.so`.
- `ldd build/cpptest`: only `libtvm_compiler.so` + `libtvm_runtime.so` +
  `libtvm_ffi.so` (no `libtvm_allvisible.so`).
- `./build/cpptest`: 144 / 144 tests pass across 29 suites.
- Smoke imports: full and `TVM_USE_RUNTIME_LIB=1` — both pass.
`TVM_USE_RUNTIME_LIB=0` correctly disables runtime-only mode (strict
parse).
- `tests/python/all-platform-minimal-test`: 75 passed, 77 skipped.
- `tests/python/runtime/`: 81 passed, 2 skipped (incl.
`test_rpc_return_remote_object` exercising the minrpc executable
end-to-end
  via `rpc.testing.GetShape{Elem,Size}`).
- `tests/python/relax/test_vm_*.py`: 150 passed, 3 deselected
(`test_vm_multi_device.py` requires 3+ GPUs; host has 2 — env, not
regression),
  2 xfailed.
- `tests/python/tirx-base/`: 273 passed, 2 skipped.
- `pre-commit` on edited files: green.

Closes #19443.
2026-04-26 07:37:01 -04:00
Shushi Hong 5c17111ed9 [Fix][Runtime][RPC] Fix remote tensor handle cleanup for RPC return values (#19410)
This PR fixes RPC tensor cleanup for tensors returned from remote calls.

When a remote function returns a `Tensor`, the RPC protocol sends both:
- the remote backing data pointer
- the remote tensor object handle used for deletion

Previously, `TensorFromRemoteOpaqueHandle` stored only the data pointer
and called
`FreeHandle(space_.data)` during local tensor destruction. That is
incorrect:
`FreeHandle` is meant for remote object handles, not raw data-space
pointers.

This could lead to invalid cleanup behavior and crashes during teardown
in RPC workflows, including the cross-compilation + RPC tutorial
scenario reported in #18923.

This change:
- stores the remote tensor object handle in `RemoteSpace`
- calls `FreeHandle(remote_tensor_handle)` during tensor destruction
- keeps cleanup fault-tolerant if the remote connection is already
closed
2026-04-16 21:39:39 +09:00
Tianqi Chen 9a8320acbd [LINT][PYTHON] Modernize annotations with ruff UP rules (#18830)
This PR enables ruff pyupgrade (UP) rules with py310 target, auto-fixing
~5600 annotation modernizations (PEP 585 generics, PEP 604 unions,
deprecated typing imports).

Also removes from __future__ import annotations from ir/module.py and
rmsnorm.py, bumps requires-python to >=3.10, and removes absolute_import
aliases from topi/contrib files.
2026-02-27 21:29:47 -05:00
Tianqi Chen 33dcea1686 [REFACTOR][LINT] Modernize ruff config (#18810)
This PR removes the extra lint violations from the codebase so lint
aligns with the latest style
2026-02-23 07:29:21 -05:00
Tianqi Chen aa2e609136 [LINT] Modernize lint to use pre-commit hooks (#18807)
This PR migrates existing lint to use pre-commit hooks
2026-02-22 11:03:21 -05:00
Tianqi Chen 543e64dbb1 [FFI][REFACTOR] Cleanup tvm_ffi python API and types (#18277)
This PR cleans up the python API to make things more consistent
with existing python array api and torch.

Device update
- device_id => index, to be consistent with torch
- device_type => dlpack_device_type() returns int
- added type property same as torch.device

API updates:

- Move the convenient method like cpu() out into tvm runtime to keep device minimal
- tvm_ffi._init_api => tvm_ffi.init_ffi_api
- tvm_ffi.register_func => tvm_ffi.register_global_func
2025-09-07 10:38:50 -04:00
Tianqi Chen 3c36ce2ec6 [FFI][REFACTOR][ABI] Rename NDArray to Tensor (#18275)
This PR Updates the NDArray => Tensor.

Both tensor and ndarray are commonly used terms.

Because the term Tensor is getting more common in the context of ML,
we do the rename to stay more aligned with torch.Tensor and DLTensor.
2025-09-06 14:33:59 -07:00
Tianqi Chen a7a0168be5 [FFI][REFACTOR] Establish tvm_ffi python module (#18226)
* [FFI][REFACTOR] Establish tvm_ffi as a standalone python module

This PR establishes tvm_ffi as a standalone python module.
The ffi is structured as a minimal pip module that can be
directly install by path or url.

examples/get_started provided a minimal example.
This is a major change as we are decoupling tvm_ffi as a
separate package, users need to install tvm_ffi separately.

Thanks to its minimal dependency, tvm_ffi can be easily installed
even just from the source by pip install ./ffi

This change would enable future improvement for library plugins
to have lightweight dependencies by just working on top of
the tvm_ffi, while the main compiler toolchain and runtime
can be layered on top.

* [FFI] Improve traceback setups

This PR improves traceback related setups
2025-08-24 15:46:20 -07:00
Tianqi Chen b8eb80b968 [FFI] Formalize ffi.Module (#18213)
This PR formalizes original runtime::Module into ffi
as ffi.Module and cleans the APIs around it.

The goal is to stablize the Module API as extra API that can benefit the overall
ffi interactions. We also refactors the c++ code that depends on the Module.
2025-08-17 23:33:05 +08:00
Tianqi Chen 4289efa0d5 [REFACTOR][PYTHON] Phase out tvm._ffi and Limited API support (#18020)
This PR phases out tvm._ffi redirections in favor of new FFI
new functions are now called via tvm.ffi.

We also enabled limited API support for python 3.12+
so the compiled binary can be forward compatible to future
python versions.
2025-05-28 16:52:36 -04:00
Tianqi Chen 95d1268982 [REFACTOR] Introduce and modernize FFI system (#17920)
This PR modernizes the FFI foundation of the project and introduce
a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi)
based on our lessons in the past few years. It implements a modern
version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md)
that unifies the packed function call and object systems.

Summary of the change:
- A dedicated clean Any/AnyView that can store strong and weak
references of items
- Function(previously PackedFunc) system built on top of the Any/AnyView
- A minimal C API that backs the overall calls. We are stabilizing the
API with a goal to bring clean, stable FFI conventions for both compiled
and registered code
- A rewrite of core python binding and generated code based on the module
- Update existing code and test cases to the new module
- Latest dlpack support
 
The new module brings many benefits thanks to the cleaner design,
to name a few:
- Any can support both POD types(int) and object types.
- Containers (e.g. Array) can now also contain Any value, e.g. now
`Array<int>` is supported, no need for boxed types
- Error handling now upgrades to object-based, allowing cleaner
traceback across languages
- Map now preserves insertion orders
- Path toward isolated stabilize minimum core ABI/API foundation module
- Type traits based design that cleanly defines how values interact
with Any system
- Automatic conversion of different types based on traits if needed 

Because FFI upgrade is at heart of the project, the change touches every
component of the system. Importantly, this is an upgrade of the ABI so the
change is not backward compatible.  The code compiled under the old
FFI won't work under the new one. We did provide example ABI translation
(e.g. LegacyTVMArgValueToFFIAny) functions for compatibility. 
The PR tries to leave files in their old places while creating redirections.
The goal is to have the first milestone landed and infrastructure in place,
so we can do further refactors to complete features and cleanup legacy code
as trackable PRs. As of now, python binding and compiled code are under the
new convention while RPC and some  other bindings still relies on legacy ABI
translation. We will work on upgrades in the coming PRs, including areas such
as reflection, phasing out legacy redirections etc.
2025-05-06 19:18:33 -04:00
Johnson Zhang 502a133429 [RPC] Fix Bug That Change Dict When Iterate The Keys 2025-04-29 06:50:45 -04:00
Tianqi Chen 660fd1e47e [DOCS] More clarity on security model of RPC server (#17382)
This PR updates the python docstrings to include
more clarity on RPC server security model.
2024-09-19 10:29:34 -07:00
Eric Lunderberg a4f20f0bbb [RPC] Raise error if server process terminated (#17101)
Prior to this PR, a local RPC server could crash without any
indication in the main process.  While typically this crash would
cause an error in the main process due to the lack of a
`RPCCode::kReturn` from the server, the delayed error can complicate
debugging.

This PR updates the local RPC server to raise an exception if the
server process returns with a non-zero exit code.
2024-06-18 13:28:15 -05:00
Ruihang Lai 5a2949bd70 Merge branch 'main' of into branch 'unity' 2024-01-18 23:09:23 -05:00
gmeeker f1bf20a950 [RPC] Fix tuning on macOS and Windows (#15771) (#16357)
* [RPC] Fix tuning on macOS and Windows (#15771)

Fix regression in (#15187) when multiprocessing start method is not 'fork',
which prevented tuning from working. This affects macOS and Windows.
Also in python 3.14 the default start method will be 'spawn'.

* [RPC] clean up _serve_loop function
2024-01-12 13:45:14 +08:00
tqchen 90c64c6dce [MERGE] Merge main into unity 2023-09-06
NOTE: use the original webgpu impl to make sure webgpu is stable.
2023-09-06 10:45:29 -04:00
Ever-Kid 512b114322 [CPP_RPC] export listdir for RPC (#15537) 2023-08-24 16:06:59 +05:30
tqchen 40300a3b92 [MERGE] Merge main into unity 2023-07-03 2023-07-03 16:29:14 -04:00
Qiang Zhang 683dfb0c04 [RPC] Report RPC Session Timeout to Client Instead of "kShutdown" (#15187)
By using RPC server in NPU board, at some time a compiled model will hang the NPU, because of the buggy operator libraries of NPU toolchain, so we must to use the session_timeout to ensure the board resource can be released by the hang jobs.

Currently the handling of session timeout error in RPC server is not good, it just kill the server loop sub process, then in the destructor of  class `RPCEndpoint` will send the code of `kShutdown` to the RPC client, but the RPC client expect receive the code of `kReturn` or `kException`, so users will see the error message that like the one reported in  https://github.com/apache/tvm/issues/15151, this error report will make users very confused and don't know what's happened.

When using tuning to search a good schedule for operators, we only want to ignore the RPC session timeout error that indicate the schedule generated is an illegal one, but other error reported by the RPC server may help us find the potential bug of our tool chain built on top of TVM, so the RPC session timeout error should be split to a standalone TVM error class.

This PR implemented these requirements by sending the RPC session timeout error message as a PRC server exception to the RPC client before kill the server loop sub process.
2023-07-01 21:06:38 -07:00
jzm-intel c07f67ceda [RPC] Disable socket SO_REUSEADDR for Windows (#15188)
This PR fix a RPC issue on Windows by disabling socket SO_REUSEADDR flag.
This flag works as expected on POSIX platforms, but will cause indeterminated behaviors on Windows sockets.
Reference: https://learn.microsoft.com/en-us/windows/win32/winsock/using-so-reuseaddr-and-so-exclusiveaddruse#using-so_reuseaddr
2023-06-30 15:59:38 -07:00
tqchen 153f70aa60 [MERGE] Merge main into unity 2023-06-01
Updated the runtime to latest
cutlass conflict resolutions
2023-06-01 19:46:25 -04:00
Krzysztof Parzyszek 4f041c96e7 [Runtime,RPC] Use f-strings for string formatting, NFC (#14967)
* [Runtime,RPC] Use f-strings for string formatting, NFC

Replace uses of % and .format() with f-strings.

Reformat modified files.

* Fix linter
2023-05-27 16:14:15 +09:00
tqchen f7eed214d1 [MERGE] Merge main into unity 2023-05-07 2023-05-07 10:53:40 -04:00
Yong Wu aa7d2bff6b [CI] Modify test cases to accommodate the CI upgrades (#14651)
* [CI] update all the images

* Update test_config

* Fix llvm_codegen_test err

* Update with newly built images

* update pylintrc

* Update i386 build

* Don't use ninja for i386

* Update torch tests

* Debug i386 platform

* check ec2 instance type for i386

* Remove gluoncv ssd example

* Update pylint

* Update test images

* Skip torch jit trace issue for arm

* Fix pylint

* update tests

* update i386 build

* update s3.py to skip non-existing files

* update pylint

* Update pylint

* Fix tests

* update clang-format to 15

* update tests for clang-format-15

* run with newly images

* skip oom test for i386

* Upgrade for DGL sample

* fix black

* Ignore a warning in doc

* New run with newly images

* Use newly generated tlcpackstaging images
2023-05-05 11:54:41 +01:00
Tianqi Chen a6831ba9c4 [Unity] Enable pod args in WebGPU (#14560)
This PR adds POD argument support in webgpu.
2023-04-10 11:13:56 -07:00
Tianqi Chen c1f304497e [Unity][WEB] Relax vm on web runtime (#14131)
This PR brings initial relax vm support on web runtime
2023-04-01 15:31:37 -04:00
Mohamad Katanbaf c2bc1ec95e [microTVM] tuning on micro targets with meta-schedule (#13514)
adds support for tuning microTVM models using meta-schedule.

Summary of the changes:

adds "c" to the targets supported by meta-schedule
implements a builder and runner for micro devices
runs a simple tuning job for verification

Co-authored-by: Mohamad <mkatanbaf@users.noreply.github.com>
2023-01-10 17:53:57 -08:00
Tianqi Chen 209845fb91 [RPC] Add fail-guard for termination time exception (#13651)
This PR adds fail-guard to reduce error messages thrown during
process termination time. Such error won't trigger test error
but will bring extra message during exit time.
2022-12-22 22:59:45 +03:00
Alexey Voronov 1b3d77a897 [RPC] Fix tracker connection termination (#13420)
* [RPC] Fix tracker connection termination

* [RPC] Unify work with random key

* additional usage of the random_key API change
2022-11-21 22:38:56 +03:00
Lite Ye da4bb4a65b Fix a typo in rpc/client.py (#12842) 2022-11-02 10:16:52 -05:00
Christian Convey 9cdc97fe23 [DLPack][runtime] Update DLPack to v0.7 (#13177)
- Update the `3rdparty/dlpack` git submodule from v0.5 to v0.7, so that
the `DLDeviceType` enumeration has an explicitly-stated underlying
storage type.  This addresses a compiler warning generated by clang
15.0.3.

- Remove `kDLHexagon` and `kDLWebGPU` from `TVMDeviceExtType`, because
those enumerators are now provided by `DLDeviceType`.

- Renumber the members of `TVMDeviceExtType` to reduce the chance of
unnoticed collision with members of `DLDeviceType`.
2022-10-31 22:21:16 -07:00
Yulv-git bdcfa01eae [Fix] Fix some typos (#11503)
Fix some typos in src/.

Co-authored-by: driazati <driazati@users.noreply.github.com>
2022-08-19 16:31:30 -07:00
Krzysztof Parzyszek d2f9f254d2 Use std::string_view, remove experimental or pre-14 variants, NFC (#12460) 2022-08-16 21:38:29 -07:00
Xiyou Zhou da2ad2d44b [RPC] Add Data & Time For RPC Tracker / Server Logging (#11950) 2022-06-29 15:07:38 -07:00
Mohamad Katanbaf 53fe596682 [rpc] Implemented rpc logging (#11232)
* Implemented rpc logging

* fixing windows build issue

* trigger

Co-authored-by: Mohamad <mkatanbaf@users.noreply.github.com>
2022-05-12 11:48:30 -07:00
Leandro Nunes ff7efe767a Revert "Implemented rpc logging (#10967)" (#11227)
This reverts commit aa3bcd9d33, because it
fails on Windows CI as reported in issue #11220. PR #11223 tries to address
it but is is failing in the regular CI with testing issue on Hexagon.
2022-05-06 12:14:20 +01:00
Mohamad Katanbaf aa3bcd9d33 Implemented rpc logging (#10967)
Co-authored-by: Mohamad <mkatanbaf@users.noreply.github.com>
2022-05-04 17:10:18 -07:00
Junru Shao ce28068da8 [MetaSchedule] Support grouping in the cost model (#10811) 2022-03-29 19:05:56 +08:00
Piotr eF 02d1539053 query rpc tracker - sort servers by key name (#10641)
* query rpc tracker - sort servers by key name

* fix black formating

Co-authored-by: pfk-beta <this_email_isnot_working@gmail.com>
2022-03-16 13:19:24 -05:00
Krzysztof Parzyszek 751f83b565 Auto-discover C/C++ compiler instead of hardcoding g++ (#10007)
Some platforms (e.g. FreeBSD) use clang as the default OS compiler,
and there is no g++.
2022-01-21 13:25:08 -08:00
Alexey Gladyshev f4dae23478 [iOS][RPC] Enable iOS simulation in public CI to cover basic tuning capabilities (#9212)
* init class for launch of server with ios simulator

* init infrastructure of tests

* added functionality for automatic loading of the simulator

* add error handling for simulator interaction

* extend tests for connection configurations

* add test for pure rpc connection

* init test for remote call

* add wrappers for connect configurations

* remove duplicate code

* add tests for simple remote call

* change policy for tests of connect configurations

* added tests to check basic functionality of rpc session

* add test for remote graph executor

* add test for auto schedule tuning

* remove hardcode parameters

* add success criterias for auto schedule tuning

* fixing problems related to running tests through the pytest

* expand the workflow for new iOS RPC tests

* update GH workflow for iOS

* update GH workflow for iOS: conda shell

* add parser for iOS RPC console log

* add depends for ios tests

* set verbose flag for rpc server

* changes related with main checkout

* add context manager class for ios rpc server launcher

* extend pythonpath

* add watchdog for start ios rpc server

* clean up GH actions workflow

* clean up GH actions workflow

* rename enum SimulatorSystem to OSName

* fix python format black

* fix bash syntax

* add doc strings for API

* skip tests, because this type of connection was broken

* fix lint

* add check that current environment has required environment variables

* code review fixes

* replaced call os.system with call subprocess.check_call

* add description for messages from iOS RPC Server
2021-10-28 09:36:41 -07:00
Alexey Gladyshev f08dca89e4 [RPC] Fix Server connecting to RPC Tracker through a Proxy (#9210) 2021-10-11 14:57:27 -04:00