github-featured/nvidia--tensorrt
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
547 Commits 42 Branches 72 Tags
v10.2.0
Commit Graph

164 Commits

Author SHA1 Message Date
Yuan Yao 2332a7191d 10.2 GA release update (#3998) 
* 10.2 GA updates

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

* update changelog

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

* revert plugin README format changes

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>

---------

Signed-off-by: Yuan Yao (yuanyao) <yuanyao@nvidia.com>
 2024-07-11 10:18:43 -07:00
Akhil Goel 21f31ba77e TensorRT OSS 10.1 GA Release 
Signed-off-by: Akhil Goel <akhilg@nvidia.com>
 2024-06-17 16:58:17 -07:00
Asfiya Baig d3107c885a TensorRT 10.0 GA Release 
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
 2024-04-30 10:00:50 -07:00
Asfiya Baig 147005ffe1 TensorRT 10.0 Release 
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
 2024-04-02 17:06:37 -07:00
Ilya Sherstyuk c89bc8368a Update TensorRT to 8.6.1 
Signed-off-by: Ilya Sherstyuk <isherstyuk@nvidia.com>
 2023-05-04 20:53:55 -07:00
tripleMu a566665884 Fix include error (#2767) 
Signed-off-by: triple-Mu <gpu@163.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 7a5c5fe1a8 Remove SD plugins 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Shuyue Lan f1a4bd3196 Plugin OSS build fixes 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Ilya Sherstyuk eb2f188c55 Run clang-format on all .cpp and .h plugin code 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Vincent Huang ebfb013b4b Enable SM-specific source lists for MHA plugins 
when TRT_LOCAL_SM is defined for develop build, we should only enable
	sm-specifc source list, so that we can reduce the build time/library size.
	I also removed the dependency on LEGACY_FUSEDEPSEP_ARCH and
	AMPERE_FUSEDEPSEP_ARCH, they are deprecated.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Morgan Huang f2a0bdecf6 Init modulatedDeformConv Plugin 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Ilya Sherstyuk a291ca6e52 Code quality improvement for various plugins 
Clean up CustomFCPlugin
Clean up CropAndResizePlugin
Clean up BatchTilePlugin
Improve code quality of proposalPlugin
Clean up DetectionLayerPlugin
Run clang-format on all .cpp and .h plugin code
upper case literal suffixes in plugin module

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 3efb69b291 Support class-independent NMS with new parameter in efficientNMSPlugin 
Also adds goldenIO testcases: one for class agnostic NMS and another for class aware NMS.
With this, golden_io_snippet.py is also modified to support checking for multiple
plugin outputs.

	Adds `class_agnostic` parameter to `EfficientNMS_TRT` plugin:

	 - Some object detection networks/architectures like YOLO series need to
	 use class-independent NMS operations.
	 - If `class_agnostic` is enabled, class-independent NMS is performed;
	 otherwise, different classes would do NMS separately.

	Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Vincent Huang 8508a9caf8 Various minor updates for plugins - part 5 
use kINT32 type for mask output in embLayerNormVarSeqlen plugin
Corrects PillarScatter supported formats
Fix bug when reading yaml file

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 451e33e107 Improve code quality on skipLayerNormPlugin 
This MR improves the code quality of skipLayerNormPlugin, including:
	 - Style: East-const, variable naming etc.
	 - Error checking/reporing: Remove aborting macros in favor of try... catch
	 and exception throwing macros
	 - Fixed-width ints

	Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Ilya Sherstyuk 9e9ed88ced Various minor updates for plugins - part 4 
Init modulatedDeformConv Plugin
fix mask input shape in readme
Document limitation of EfficientNMS_ONNX_TRT plugin
Add missing license header to loggerFinder.h

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Micah Villmow 93059a5ad1 fix broken namespaces in the plugins 
The plugins belong in the nvinfer1::plugin namespace.

	However some plugins were not only polluting the global namespace, but
	were exposing the same function twice! Once in the global namespace and
	once in the nvinfer1::plugin namespace. This fixes this by pushing
	everything into the nvinfer1::plugin namespace.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Shuyue Lan 3bd12e2bcb relocate LoggerFinder to NvInferRuntime.h 
Relocate LoggerFinder to NvInferRuntime.h and make it an Iclass.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Ilya Sherstyuk 6e3eac473c Various minor updates for plugins - part 3 
Shorter plugin attribute test definitions
Fix incorrect names of some deserialize methods
Memory issues in PriorBoxPlugin

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Oxana Korzh f6fdb268b7 fix remaining copyrights to 2023 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 7ea223a874 Fix deserialization issues in a couple of plugins 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Micah Villmow 0c757dd67f cppcheck fixes part 7 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Oxana Korzh 8630027467 Fix vc-plugin exports and add missing TENSORRTAPI for windows build 
- Fix vc-plugin exports to have only entry point symbols
- Add missing TENSORRTAPI for windows build

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Vincent Huang 1759be6520 update after review 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:29 -07:00
Samurdhi Karunaratne 9a69a15643 Move BERT plugins to nvinfer1::plugin 
Signed-off-by: Samurdhi Karunaratne <skarunaratne@nvidia.com>

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:28 -07:00
Micah Villmow c27025ffd2 cppcheck fixes part 4 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:28 -07:00
Micah Villmow 69ccf78894 fix a broken fix to use divUp and not roundUp. 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:28 -07:00
Micah Villmow 27e9608b4a add LayerNormPlugin 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-16 20:53:28 -07:00
Oxana Korzh c6d301507a Various minor updates for plugins - part 2 
Remove pointer hashing in testing.
fixed warnings reported by clang-tidy bugprone-macro-parentheses
fixed warnings reported by clang-tidy misc-throw-by-value-catch-by-reference
Rename from VFC -> VC

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:50 -07:00
Shuyue Lan d539ed98fe add APIs for loading plugins from buffers or lib files 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:50 -07:00
Tal Cherckez 3722b38afd Various minor updates for plugins - part 1 
Update datatype switch statements for kQUANT_E4M3
Fix win CUDA12.0 build

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Shuyue Lan 3d7ec39cd5 add getPluginCreators and LoggerFinder for vfc plugin 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Jin Li (Engrg-Hardware 1) e8d5e1671f Various minor fixes for plugins 
Limit output_dims == W[1] for plugin test
update embLayernorm plugin doc
Remove else after return as they are not required.
Fix FlattenConcat plugin readme

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Zhihan Jiang fde09d7530 Replace IMMA/HMMA fmha_v2 with IGMMA/HGMMA ones (missing s=64/96) 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Hongyu Miao efaf76b2eb fixed warnings reported by Clang-tidy bugprone-branch-clone 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Shuyue Lan 025060405f create a shared library that contains ROIAlign plugin 
For version forward compatibility, we want to generate a shared library which contains vfc plugins. Currently we only have ROIAlign plugin in the list.

Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Samurdhi Karunaratne 8de77781b5 use kINT32 for BERT MHA plugin mask 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Tal Cherckez 633401abb4 Fix incorrect asserts 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Shuyue Lan 3ec7b425cf Cherry-pick sample changes and update headers & python bindings from TRT rel-8.6 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Samurdhi Karunaratne f98a1b2e33 Add exports.def for OSS Windows build of plugins 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Rajeev Rao 2c1bceef22 Cherry-pick Enable functionality of epsilon in groupNormPlugin 
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2023-03-15 17:38:49 -07:00
Xiaodong (Vincent) Huang 8e756f163f Cherry-pick fix fcPlugin build failure on CUDA12 (#2743) 
Signed-off-by: Vincent Huang <vincenth@nvidia.com>
 2023-03-07 19:42:46 -08:00
asfiyab-nvidia b0c259a749 TensorRT OSS - 8.5.3.1 Release (#2630) 
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
 2023-02-03 12:26:51 -08:00
Yuan Yao 8423367cfc cherry-pick QKVToContext plugin changes (#2548) 
Signed-off-by: Yuan Yao <yuanyao@nvidia.com>
 2022-12-19 11:55:52 -08:00
Julien Demouth 85f59aa72b Update seq2len plugin to support dynamic shapes 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-07 22:12:40 -08:00
Rajeev Rao e82aa6a836 Refactor fMHA and fMHCA plugins 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-07 22:12:40 -08:00
Nikita Korobov 6f321e9334 Updated fMHCA cubins 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-07 22:12:40 -08:00
Samurdhi Karunaratne cf43abd8e4 Refactor seqLen2SpatialPlugin plugin code to meet TRT guidelines 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-06 21:10:47 -08:00
Rajeev Rao a02fc46ae8 Refactor groupNormPlugin code 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-06 21:10:47 -08:00
Julien Demouth 718670ab60 demoDiffusion - Enable Turing optimizations + new image sizes 
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
 2022-12-02 07:59:29 -08:00