Shorter plugin attribute test definitions
Fix incorrect names of some deserialize methods
Memory issues in PriorBoxPlugin
Signed-off-by: Shuyue Lan <shuyuel@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
Update tools including onnx-graphsurgeon(v0.3.25), Polygraphy(v0.43.1), tensorflow-quantization(v0.2.0)
Update onnx-tensorrt(8.5GA)
Mass merge sample, plugin, demo, python changes for TRT-8.5 GA release
Update demoBERT benchmark data for TensorRT 8.5.1
Signed-off-by: Simeng Liu <simengl@nvidia.com>
Signed-off-by: Simeng Liu <simengl@nvidia.com>
This graceful failure is needed to enable tests under `trt_mod_test`.
`PLUGIN_ASSERT` is used in many places inside plugin methods, most of
which are `noexcept` and do not contain try-catch blocks. So changing
`PLUGIN_ASSERT` would require a major refactor which is beyond
rel-8.4. Instead, this MR introduces `PLUGIN_VALIDATE` as a temporary
replacement for `PLUGIN_ASSERT` in places where it is required to enable
valid/invalid attribute tests (mostly `createPlugin` and plugin constructors).
i.e. `PLUGIN_VALIDATE` implements the expected behavior of `PLUGIN_ASSERT`
(throws) while `PLUGIN_ASSERT` still aborts.
Signed-off-by: samurdhi karunaratne <skarunaratne@nvidia.com>
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
PriorBox plugin serialize CPU metadata (array size) A and GPU data
(array elements) B' in engine. B' is modified from CPU array B
when constructing the object. A deserialized object then holds data
A and B' which is different from the original (A and B).
If a new object is created from a deserialized one via `PriorBox::clone()`,
which rebuilds array elements at GPU side from CPU holding array A and
B', the generated GPU data is incorrect (A and B''), resulting in
wrong inference result.
As PriorBox is designed to track data in specific format, we now
serialize only the CPU data A and B, i.e. the parameters that used to
construct a PriorBox object, to engine.
bad image processing with deserialized engine
1. Fixed the memory deallocation error in plugin PriorBox::clone() method
even without serialization by initializing empty pointer to nullptr.
2. Initialized weights to empty structs
3. Added mParam.aspectRatios to serialization and deserialization since
mParam.aspectRatios are different from aspectRatios device weights in count and values.
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
Kevin Chen