167 Commits

Author SHA1 Message Date
poweiw c664ce22b0 Update for 10.4-GA
Signed-off-by: poweiw <poweiw@nvidia.com>
2024-09-11 17:26:50 -07:00
Balarama Raju Buddharaju 84dd6ed18c Updates for Release 10.3-GA.
Signed-off-by: Balarama Raju Buddharaju <bbuddharaju@nvidia.com>
2024-08-08 12:32:01 -07:00
Yuan Yao (yuanyao) 3cfd5a3e39 add missing IN plugin files
Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>
2024-07-31 10:13:29 -07:00
Yuan Yao 2332a7191d 10.2 GA release update (#3998)
* 10.2 GA updates

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

* update changelog

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

* revert plugin README format changes

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

---------

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>
2024-07-11 10:18:43 -07:00
Akhil Goel 21f31ba77e TensorRT OSS 10.1 GA Release
Signed-off-by: Akhil Goel <akhilg@nvidia.com>
2024-06-17 16:58:17 -07:00
Asfiya Baig d3107c885a TensorRT 10.0 GA Release
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
2024-04-30 10:00:50 -07:00
Asfiya Baig 147005ffe1 TensorRT 10.0 Release
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
2024-04-02 17:06:37 -07:00
Ilya Sherstyuk c89bc8368a Update TensorRT to 8.6.1
Signed-off-by: Ilya Sherstyuk <isherstyuk@nvidia.com>
2023-05-04 20:53:55 -07:00
tripleMu a566665884 Fix include error (#2767)
Signed-off-by: triple-Mu <gpu@163.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 7a5c5fe1a8 Remove SD plugins
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Shuyue Lan f1a4bd3196 Plugin OSS build fixes
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Ilya Sherstyuk eb2f188c55 Run clang-format on all .cpp and .h plugin code
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Vincent Huang ebfb013b4b Enable SM-specific source lists for MHA plugins
when TRT_LOCAL_SM is defined for develop build, we should only enable
	sm-specifc source list, so that we can reduce the build time/library size.
	I also removed the dependency on LEGACY_FUSEDEPSEP_ARCH and
	AMPERE_FUSEDEPSEP_ARCH, they are deprecated.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Morgan Huang f2a0bdecf6 Init modulatedDeformConv Plugin
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Ilya Sherstyuk a291ca6e52 Code quality improvement for various plugins
Clean up CustomFCPlugin
Clean up CropAndResizePlugin
Clean up BatchTilePlugin
Improve code quality of proposalPlugin
Clean up DetectionLayerPlugin
Run clang-format on all .cpp and .h plugin code
upper case literal suffixes in plugin module

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 3efb69b291 Support class-independent NMS with new parameter in efficientNMSPlugin
Also adds goldenIO testcases: one for class agnostic NMS and another for class aware NMS.
With this, golden_io_snippet.py is also modified to support checking for multiple
plugin outputs.

	Adds `class_agnostic` parameter to `EfficientNMS_TRT` plugin:

	 - Some object detection networks/architectures like YOLO series need to
	 use class-independent NMS operations.
	 - If `class_agnostic` is enabled, class-independent NMS is performed;
	 otherwise, different classes would do NMS separately.

	Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Vincent Huang 8508a9caf8 Various minor updates for plugins - part 5
use kINT32 type for mask output in embLayerNormVarSeqlen plugin
Corrects PillarScatter supported formats
Fix bug when reading yaml file

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 451e33e107 Improve code quality on skipLayerNormPlugin
This MR improves the code quality of skipLayerNormPlugin, including:
	 - Style: East-const, variable naming etc.
	 - Error checking/reporing: Remove aborting macros in favor of try... catch
	 and exception throwing macros
	 - Fixed-width ints

	Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Ilya Sherstyuk 9e9ed88ced Various minor updates for plugins - part 4
Init modulatedDeformConv Plugin
fix mask input shape in readme
Document limitation of EfficientNMS_ONNX_TRT plugin
Add missing license header to loggerFinder.h

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Micah Villmow 93059a5ad1 fix broken namespaces in the plugins
The plugins belong in the nvinfer1::plugin namespace.

	However some plugins were not only polluting the global namespace, but
	were exposing the same function twice! Once in the global namespace and
	once in the nvinfer1::plugin namespace. This fixes this by pushing
	everything into the nvinfer1::plugin namespace.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Shuyue Lan 3bd12e2bcb relocate LoggerFinder to NvInferRuntime.h
Relocate LoggerFinder to NvInferRuntime.h and make it an Iclass.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Ilya Sherstyuk 6e3eac473c Various minor updates for plugins - part 3
Shorter plugin attribute test definitions
Fix incorrect names of some deserialize methods
Memory issues in PriorBoxPlugin

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Oxana Korzh f6fdb268b7 fix remaining copyrights to 2023
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 7ea223a874 Fix deserialization issues in a couple of plugins
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Micah Villmow 0c757dd67f cppcheck fixes part 7
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Oxana Korzh 8630027467 Fix vc-plugin exports and add missing TENSORRTAPI for windows build
- Fix vc-plugin exports to have only entry point symbols
- Add missing TENSORRTAPI for windows build

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Vincent Huang 1759be6520 update after review
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 9a69a15643 Move BERT plugins to nvinfer1::plugin
Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:28 -07:00
Micah Villmow c27025ffd2 cppcheck fixes part 4
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:28 -07:00
Micah Villmow 69ccf78894 fix a broken fix to use divUp and not roundUp.
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:28 -07:00
Micah Villmow 27e9608b4a add LayerNormPlugin
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-16 20:53:28 -07:00
Oxana Korzh c6d301507a Various minor updates for plugins - part 2
Remove pointer hashing in testing.
fixed warnings reported by clang-tidy bugprone-macro-parentheses
fixed warnings reported by clang-tidy misc-throw-by-value-catch-by-reference
Rename from VFC -> VC

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:50 -07:00
Shuyue Lan d539ed98fe add APIs for loading plugins from buffers or lib files
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:50 -07:00
Tal Cherckez 3722b38afd Various minor updates for plugins - part 1
Update datatype switch statements for kQUANT_E4M3
Fix win CUDA12.0 build

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Shuyue Lan 3d7ec39cd5 add getPluginCreators and LoggerFinder for vfc plugin
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Jin Li (Engrg-Hardware 1) e8d5e1671f Various minor fixes for plugins
Limit output_dims == W[1] for plugin test
update embLayernorm plugin doc
Remove else after return as they are not required.
Fix FlattenConcat plugin readme

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Zhihan Jiang fde09d7530 Replace IMMA/HMMA fmha_v2 with IGMMA/HGMMA ones (missing s=64/96)
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Hongyu Miao efaf76b2eb fixed warnings reported by Clang-tidy bugprone-branch-clone
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Shuyue Lan 025060405f create a shared library that contains ROIAlign plugin
For version forward compatibility, we want to generate a shared library which contains vfc plugins. Currently we only have ROIAlign plugin in the list.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Samurdhi Karunaratne 8de77781b5 use kINT32 for BERT MHA plugin mask
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Tal Cherckez 633401abb4 Fix incorrect asserts
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Shuyue Lan 3ec7b425cf Cherry-pick sample changes and update headers & python bindings from TRT rel-8.6
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Samurdhi Karunaratne f98a1b2e33 Add exports.def for OSS Windows build of plugins
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Rajeev Rao 2c1bceef22 Cherry-pick Enable functionality of epsilon in groupNormPlugin
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2023-03-15 17:38:49 -07:00
Xiaodong (Vincent) Huang 8e756f163f Cherry-pick fix fcPlugin build failure on CUDA12 (#2743)
Signed-off-by: Vincent Huang <vincenth@nvidia.com>
2023-03-07 19:42:46 -08:00
asfiyab-nvidia b0c259a749 TensorRT OSS - 8.5.3.1 Release (#2630)
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
2023-02-03 12:26:51 -08:00
Yuan Yao 8423367cfc cherry-pick QKVToContext plugin changes (#2548)
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
2022-12-19 11:55:52 -08:00
Julien Demouth 85f59aa72b Update seq2len plugin to support dynamic shapes
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2022-12-07 22:12:40 -08:00
Rajeev Rao e82aa6a836 Refactor fMHA and fMHCA plugins
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2022-12-07 22:12:40 -08:00
Nikita Korobov 6f321e9334 Updated fMHCA cubins
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
2022-12-07 22:12:40 -08:00