- enable fp16 precision in multilevelCropAndResizePlugin and multilevelProposeROIPlugin
- Algorithms optimization for NMS kernels and ROIAlign kernel
- Fix invalid cuda config issue when bs is larger than 32
- Fix issues found on Jetson NANO
Signed-off-by: Tyler Zhu <tylerz@nvidia.com>
1. add varlen mha fp16 slen=384 kernel for sm_86
2. referesh all sm_86 kernels now use NVCC -gencode=arch=compute_86,code=\"sm_86\"
3. use unfused kernel for fixed len s=384 fp16
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
PriorBox plugin serialize CPU metadata (array size) A and GPU data
(array elements) B' in engine. B' is modified from CPU array B
when constructing the object. A deserialized object then holds data
A and B' which is different from the original (A and B).
If a new object is created from a deserialized one via `PriorBox::clone()`,
which rebuilds array elements at GPU side from CPU holding array A and
B', the generated GPU data is incorrect (A and B''), resulting in
wrong inference result.
As PriorBox is designed to track data in specific format, we now
serialize only the CPU data A and B, i.e. the parameters that used to
construct a PriorBox object, to engine.
bad image processing with deserialized engine
1. Fixed the memory deallocation error in plugin PriorBox::clone() method
even without serialization by initializing empty pointer to nullptr.
2. Initialized weights to empty structs
3. Added mParam.aspectRatios to serialization and deserialization since
mParam.aspectRatios are different from aspectRatios device weights in count and values.
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
In groupNormalizationPlugin, the .cu files are not added to
PLUGIN_CU_SOURCES, contrary to other plugins.
This is problematic because the NVCC flags are not set correctly when
compiling those .cu files, e.g. GENCODES.
While this is not a direct issue for this plugin in particular, it
should be fixed as the documentation refers to it for building new
plugings.
Signed-off-by: Mickaël Seznec <mickael.seznec@gmail.com>
- enable n-bit radix sort for NMS_TRT plugin
- implement NMSDynamic_TRT plugin and enable n-bit radix sort for it
- enable n-bit radix sort for BatchedNMS_TRT and BatchedNMSDynamic_TRT plugins
- fixed a bug in configurePlugin() method of BatchedNMS_TRT
- other minor fixes for BatchedNMS_TRT/BatchedNMSDynamic_TRT plugins
Signed-off-by: Rajeev Rao <rajeevrao@nvidia.com>
The original implementation results in wrong results of sum of softmax such that the results of BERT models (128 < seq_len < 384 and seq_len > 384) are very large or even 'nan'.
This implementation fix the computational problem such that the results of BERT models (128 < seq_len < 384 and seq_len > 384) become correct.
Signed-off-by: yuanzexi <hiyuanzexi@outlook.com>
Kevin Chen