## Summary
This PR completes the TFLite `NON_MAX_SUPPRESSION_V5` implementation in
Relax by adding support for `soft_nms_sigma != 0`.
It extends `relax.vision.non_max_suppression` with soft-NMS attributes,
updates the TFLite frontend to consume the soft-NMS outputs correctly,
and aligns the TOPI implementation with LiteRT's reference behavior.
Relates to #19412.
## Changes
1. **Relax / TOPI soft-NMS support**
- Extend `NonMaximumSuppressionAttrs` with `soft_nms_sigma` and
`score_threshold`.
- Thread the new attributes through Relax op registration, Python
wrapper, and legalization.
- Add soft-NMS handling to TOPI classic NMS so
`relax.vision.non_max_suppression` can represent the
`NON_MAX_SUPPRESSION_V5` behavior.
2. **TFLite frontend support for `NON_MAX_SUPPRESSION_V5`**
- Remove the previous `soft_nms_sigma != 0` unsupported-path guard in
the TFLite frontend.
- Forward `soft_nms_sigma` and `score_threshold` into
`relax.vision.non_max_suppression`.
- Handle the soft-NMS return path explicitly so the frontend reads
decayed scores from the processed NMS output instead of re-reading the
original score tensor.
3. **Soft-NMS correctness fixes**
- Fix the soft-NMS path so boxes whose scores fall below the threshold
after decay are invalidated consistently.
- Keep returned indices and decayed scores aligned in both the TOPI TIR
implementation and the NumPy reference implementation.
- Update the soft-NMS candidate selection logic to re-pick the current
best candidate after each decay step, matching LiteRT's
reference behavior.
- Align the Gaussian decay formula with LiteRT.
4. **Test coverage**
- Add Relax tests for soft-NMS struct-info inference and legalization.
- Add Relax E2E tests covering reordered outputs after score decay and
other soft-NMS follow-up cases.
- Add TFLite frontend tests for `NON_MAX_SUPPRESSION_V5` with
`soft_nms_sigma != 0`.
- Add IR checks to verify that `soft_nms_sigma` and `score_threshold`
are forwarded correctly.
## Testing
```bash
python -m pytest -n 1 tests/python/relax/test_op_vision.py -k "all_class_non_max_suppression or get_valid_counts or nms" -v
python -m pytest tests/python/relax/test_frontend_tflite.py -k "nms_v5" -v
```
## Result:
- Relax vision tests passed locally
- TFLite `NON_MAX_SUPPRESSION_V5` coverage added for both hard-NMS and
soft-NMS paths
## Problem
`tir.round` constant-folds using `std::nearbyint` (IEEE 754
ties-to-even), but all backends lower it to platform `round()` which
uses ties-away-from-zero. This means compiled code can produce different
results from constant-folded code for midpoint values:
| Input | Constant-fold (ties-to-even) | Compiled (ties-away) |
|-------|-----|------|
| 0.5 | 0.0 | 1.0 |
| 2.5 | 2.0 | 3.0 |
| -0.5 | 0.0 | -1.0 |
This was identified as a follow-up to #19367 — see [this
comment](https://github.com/apache/tvm/pull/19367#issuecomment-4201800320).
## Fix
Align all backends to use ties-to-even intrinsics, matching the
constant-folding behavior:
| Backend | Before | After |
|---------|--------|-------|
| LLVM/ROCm/Hexagon | `llvm::Intrinsic::round` |
`llvm::Intrinsic::nearbyint` |
| NVPTX | `__nv_round[f]` | `__nv_nearbyint[f]` |
| CUDA | `round`/`roundf` | `nearbyint`/`nearbyintf` (f16/bf16 already
used `hrint`) |
| Metal/OpenCL | `round` | `rint` |
| Vulkan/SPIR-V | `GLSLstd450Round` | `GLSLstd450RoundEven` |
Also fixes OpenCL codegen where `tir.nearbyint` was incorrectly mapped
to OpenCL `round()` instead of `rint()`.
Updates `op.h` documentation to explicitly state ties-to-even semantics
for both `round()` and `nearbyint()`.
## Testing
```
python -m pytest tests/python/tirx-base/test_tir_intrin.py -xvs
```
New `test_round_ties_to_even` verifies midpoint inputs `[0.5, 1.5, 2.5,
3.5, -0.5, -1.5, -2.5, -3.5]` produce ties-to-even results on the LLVM
backend. All 12 tests pass (10 passed, 2 skipped for CUDA).
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
## Summary
Add `relax.vision.get_valid_counts` and classic
`relax.vision.non_max_suppression` for object-detection post-processing
pipelines.
`get_valid_counts` performs score-based bounding box filtering and
compacts valid boxes to the front of each batch. Classic
`non_max_suppression` performs flexible IoU-based suppression on
filtered boxes, complementing existing `all_class_non_max_suppression`
for custom post-processing workflows.
This PR implements the Relax-level registration, legalization, TOPI
compute, and test coverage for both operators.
## Changes
**Relax op registration and legalization:**
- C++ op functions, FFI registration, and struct info inference for both
operators (`vision.h`, `vision.cc`)
- Python wrappers with Relax docstrings (`vision.py`)
- Legalization to `topi.vision.get_valid_counts` and
`topi.vision.non_max_suppression`
- Additional struct-info validation for `score_index`, `id_index`, and
`coord_start` when `elem_length` is statically known
**TOPI and testing:**
- Full TOPI implementation for `get_valid_counts`
- Reimplementation of classic `non_max_suppression` in TOPI
- NumPy reference implementations in `tvm.topi.testing` for both
operators
- Op-level tests for struct info inference, legalization, invalid
attribute ranges, and e2e numerical correctness
- Stronger legalization tests that verify both `relax.call_tir`
introduction and removal of the original Relax vision op
## Limitations
- Attribute range validation for `score_index`, `id_index`, and
`coord_start` is only enforced when the input `elem_length` is
statically known during struct-info inference.
- Classic `non_max_suppression` follows the existing Relax / TOPI API
shape and is intended for single-class or class-aware custom
post-processing flows, distinct from `all_class_non_max_suppression`.
## Validation
```bash
pytest tests/python/relax/test_op_vision.py -k "get_valid_counts" -v
pytest tests/python/relax/test_op_vision.py -k "test_nms_" -v
```
All related tests passed.
## Summary
Add Relax `roi_align` support and wire it through the ONNX and PyTorch
frontends.
## Changes
- add `relax.vision.roi_align`, including attrs, Python wrapper, struct
info inference, and legalization
- add TOPI `roi_align` compute and keep both legacy and aligned ROIAlign
semantics
- support ONNX `RoiAlign`, including `coordinate_transformation_mode`
handling for `output_half_pixel` and `half_pixel`
- support PyTorch `torchvision.ops.roi_align` in the exported-program
frontend, including the `aligned` flag
- add regression tests for Relax op inference, legalization, TVMScript
parsing, ONNX frontend import, and PyTorch frontend import
- add aligned ROIAlign test coverage to make sure sub-pixel RoIs no
longer use the legacy `min=1.0` clamp
## Validation
- `pytest tests/python/relax/test_op_vision.py -k roi_align`
- `pytest tests/python/relax/test_tvmscript_parser_op_vision.py -k
roi_align`
- `pytest tests/python/relax/test_frontend_onnx.py -k roi_align`
- `pytest tests/python/relax/test_frontend_from_exported_program.py -k
roi_align`
This PR completes the Relax/ONNX/Torch roi_align work tracked in #18928.
This PR enables ruff pyupgrade (UP) rules with py310 target, auto-fixing
~5600 annotation modernizations (PEP 585 generics, PEP 604 unions,
deprecated typing imports).
Also removes from __future__ import annotations from ir/module.py and
rmsnorm.py, bumps requires-python to >=3.10, and removes absolute_import
aliases from topi/contrib files.
## Summary
The int_div optimization in `topi.image.resize` was applied
unconditionally
for `nearest_neighbor` + `asymmetric` mode, regardless of rounding
method.
This caused accuracy issues when `rounding_method` is not `"floor"`
(e.g.,
`"round"`, `"round_prefer_ceil"`), because integer division truncates
toward
zero rather than rounding.
**Fix**: Gate the int_div optimization on `rounding_method == "floor"`
or
`rounding_method == ""` (the default, which gets resolved to `"floor"`
for
non-align_corners modes).
- Updates `_resize_2d` in `python/tvm/topi/image/resize.py`
- Updates reference implementation in
`python/tvm/topi/testing/resize_python.py`
- Updates legalize test expected output to reflect the new behavior
This PR modernizes the FFI foundation of the project and introduce
a new minimal and lightweight module [tvm ffi](https://github.com/apache/tvm/tree/refactor-s3/ffi)
based on our lessons in the past few years. It implements a modern
version of the [Unified Packed and Object RFC](https://github.com/apache/tvm-rfcs/blob/main/rfcs/0097-unify-packed-and-object.md)
that unifies the packed function call and object systems.
Summary of the change:
- A dedicated clean Any/AnyView that can store strong and weak
references of items
- Function(previously PackedFunc) system built on top of the Any/AnyView
- A minimal C API that backs the overall calls. We are stabilizing the
API with a goal to bring clean, stable FFI conventions for both compiled
and registered code
- A rewrite of core python binding and generated code based on the module
- Update existing code and test cases to the new module
- Latest dlpack support
The new module brings many benefits thanks to the cleaner design,
to name a few:
- Any can support both POD types(int) and object types.
- Containers (e.g. Array) can now also contain Any value, e.g. now
`Array<int>` is supported, no need for boxed types
- Error handling now upgrades to object-based, allowing cleaner
traceback across languages
- Map now preserves insertion orders
- Path toward isolated stabilize minimum core ABI/API foundation module
- Type traits based design that cleanly defines how values interact
with Any system
- Automatic conversion of different types based on traits if needed
Because FFI upgrade is at heart of the project, the change touches every
component of the system. Importantly, this is an upgrade of the ABI so the
change is not backward compatible. The code compiled under the old
FFI won't work under the new one. We did provide example ABI translation
(e.g. LegacyTVMArgValueToFFIAny) functions for compatibility.
The PR tries to leave files in their old places while creating redirections.
The goal is to have the first milestone landed and infrastructure in place,
so we can do further refactors to complete features and cleanup legacy code
as trackable PRs. As of now, python binding and compiled code are under the
new convention while RPC and some other bindings still relies on legacy ABI
translation. We will work on upgrades in the coming PRs, including areas such
as reflection, phasing out legacy redirections etc.
This PR starts the step 0 to phase out relay from the current
development main branch. This PR focuses on the python
components of relay, autotvm, auto_scheduler. To make the change
manageable, we will also do followup steps on te.Schedule and
c++ components in followup PRs.
To continue support community members who depends on
legacy flows, the [v0.19.0](https://github.com/apache/tvm/tree/v0.19.0)
branch will continue contain these components.
As noted in [discussion on phasing out legacy components](https://discuss.tvm.apache.org/t/phasing-out-legacy-components/17703/30),
this would help us to do two purposes:
- By removing outdated or redundant elements, we can significantly
reduce complexity and improve maintainability.
- Unify our focus: Concentrating our efforts on the new unity flow
will allow for more efficient development and innovation.
It is also a good opportunity for us to revisit and reduce CI time.
The past relay legacy flow contains a lot of end to end tests that
requires hardware resources to run and causing long CI time.
Moving onwards, we can focus more on unit-tests that focuses
on structural equality and runs within seconds, while be mindful
about tests that requires hardware resources (by restricting them
to specific folders and CI nightly in some cases).
---
Co-authored-by: Siyuan Feng <hzfengsy@sjtu.edu.cn>
This PR adds support for group_conv1d_transpose_ncw generic.
* Merge commit '803c4ad0847f492491d8714c7ab6f52c679e6431'
* apply black format
* skip test for cuda (unimplemented)
* avoid code duplication for conv1d_transpose_ncw
* rename func & cleanup
---------
Co-authored-by: jonghewk <jonghewk@rebellions.ai>
Some versions of numpy disallow the following:
>>> import numpy as np
>>> a = np.zeros(10)
>>> b = [slice(None)]
>>> a[b]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis
(`None`) and integer or boolean arrays are valid indices
When b is a tuple, it works fine:
>>> a[tuple(b)]
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
This pr modifies the topi implementation (which is also the legalizer's backend of Relax) of LayerNorm and GroupNorm operators to allow them to accept fp16 inputs, cast to fp32 internally, and produce fp16 outputs.
This can help eliminate unnecessary casts caused by AMP.
As more and more ML models nowadays contain the group normalization
computation, we find it beneficial to introduce this op to TOPI level.
It will enable us to optimize the group normalization operation as a
whole in a more convenient way.
This PR introduces the group normalization op to TOPI. The group norm
operation was introduced in https://arxiv.org/abs/1803.08494. The
implementation uses tuple reduction, same as the implementation of layer
norm. Implemented with tuple reduction, the corresponding generated TIR
function can be optimized by cross-thread reduction or rfactor through
MetaSchedule.
Co-authored-by: Bohan Hou <spectrometerh@gmail.com>
Prior to this PR, TOPI batch_norm only supports inference.
This PR adds training: bool flag and momentum: float argument to support training mode (update moving_mean / var and return), which aligns with torch.nn.functional.batch_norm.
Pytorch's grid_sample() supports various interpolation options:
(1) data dimension: 2D / 3D
(2) interpolation method: nearest / bilinear / bicubic
(3) padding_mode: zeros / border / reflection
(4) align_corners: True / False
However, TVM only supports a part of above options:
(1) data dimension: 2D
(2) interpolation method: bilinear
(3) padding_mode: zeros / border
(4) align_corners: True
This commit completes the options not supported by TVM, and keeps existing
grid_sample of onnx/pytorch uninfluenced.
Co-authored-by: shukun.net
* Make all required adjusts in the code to comply with the new version
* Upadte ci-lint to v0.71, based on tlcpackstaging/ci_lint:20220411-060305-45f3d4a52
* [TOPI] Add support for groupped conv3d
Change conv3d to use generic conv implementation which supports groupped
convolutions. Also, remove support for non-float16 tensorcore operations
as they cause large degradation in accuracy. Generic conv now supports
autoscheduler.
* correct none check
* add tests for floordiv simplification
* fixed incorrect test for autoscheduler
* formatting
* add groups to winograd
* fix tensorcore
* manually simplify index instead of relying on simplifier
* formatting
* add groups argument to conv3d_ncdhw_winograd_without_weight_transform
* formatting
* [relay] Fix stack overflow in device_planner observed on windows due to recursive function calls.
* Revert "[relay] Fix stack overflow in device_planner observed on windows due to recursive function calls."
This reverts commit 70581364771e2415b37b202a9fa6a937f275cfc6.
* [PyTorch] Add grid_sample with zeros and border padding mode for PyTorch.
* [CUTLASS] Add wgrad support (without split-k)
* run black
* wgrad tests now work under pytest
* dw conv2d properly supported for wgrad
* all tests work
* fixed for sm75
* cpplint
* fix conv2d grad test
* [TOPI] Support grouped conv1d
Generalize the conv2d compute statement to a generic convNd that
supports any layout and groups. Replace some existing conv2d and conv1d
compute statements with this generic compute. Also add a topi
group_conv1d compute that uses the generic convNd compute. Existing
schedules for conv1d work with group_conv1d, so they are reused.
* permute reduction axis order
* formatting
* Add topi batch norm and tests
* Handle none values correctly
* Return correct nun outputs for onnx
* Use moving var/mean and update tests
* Add a test for batch norm folding
* Fix comment
* Format with black
* Re-order test args to match interface
* Call fold constant manually
* f wrong type check in conv2d_transpose
* add test case for conv2d transpose
* add groups support for conv2d_transpose
* add naive implementation and schedule for conv2d with groups
* enable tests for cpu and arm_cpu, raise error for cuda platform
* revert the cuda and generic strategy
* revert back the x86 strategy
* revert back the arm_cpu strategy
* revert back the arm_cpu strategy
* revert back the arm_cpu strategy
* fix EOF of x86
* clang lint updated c++ code
* update topi implementation
* Revert test
* Revert test
* add generic/x86/arm specialization for conv2d_transpose with groups > 1
* remove commentted codes
* fix lint
* fix lint
* fix c++ lint
* fix lint
* fix python lint
* remove comments and reformat
* lint file
* lint code
* fix lint
* update logging information in convolution.h
Co-authored-by: Alicja Kwasniewska <alicja.kwasniewska@sima.ai>
* Add relay definition
* 1D cpu test working
* multi dim working
* gpu version working
* check shape in type rel
* support side
* use target specfic max threads
* add relay boilerplate
* relay test working
* cleanup topi test
* fix test
* add torch converter
* handle other cases
* more topi test
* support torch bucketize
* update doc
* fix tests
* fix lint
* rebase fix
* make the test case smaller
* add tests for edge cases
* replace "side" attribute with boolean "right"
* add more descrition to binear_search IR gen params
* return index from binary_search rather than update inplace
* remove unused argument
* format fix
* [Topi][Testing] Minor cleanup for python reference implementations
- Use input dtype for dilate/conv2d accumulate in python
impl. Previously, the python implementations of dilation and conv2d
would use numpy default dtype in some cases, rather than the input
data's dtype.
- Added fallback for datatypes not supported by scipy.signal.convolve2d (e.g. float16).
- Refactored to avoid duplication, use common get_pad_tuple functionality.
* [Topi][UnitTests] Added float16 tests to test_topi_dense.py
* [Topi][UnitTests] Added float16 to test_topi_conv2d_nchw.py
* [Topi][Float16] Added float16 tests for depthwise conv2d.
* [UnitTests] Explicitly set seed for float16 tests
Intended to avoid flaky test failures later due to rounding errors.
* [UnitTests] Fixed a few failing unit tests.
- ref_data must be a test fixture, not acquired through
request.getfixturevalue, in order to have the random_seed be known.
- dilate_python's return value didn't follow `out_dtype`.
- The test_topi_conv3d tests had the reference results computed in
float64, due to dilate_python() not respecting the input data type.
With the correct dtype, the tolerances needed to be slightly widened.
Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
* Add basic support for batch matmul transpose
* Update
* Lint fix & add tf convert support
* Update
Lint fix
* Bug fix for qnn.batch_matmul
* Bug fix for tensorflow test
* Add grad support for batch_matmul
* Lint fix
Re-triggle CI
Bug fix
Re-triggle CI
Re-triggle CI
Re-triggle CI
ConvolutedDog