Asfiya BaigskipLayerNormPlugin
Table Of Contents
Description
Adds a residual tensor, applies layer normalization, i.e., transforms the mean and standard deviation to beta and gamma respectively. Optionally, adds a bias vector before layer-normalization.
Structure
The skipLayerNormPlugin takes two inputs; input and skip.
input
For V1 and V2, input is a tensor with shape [S, B, E, 1, 1] where S is the sequence length, B is the batch size, E is the hidden size, and the last two dimensions are of size 1.
For V3 and V4, input is a tensor with shape [1, E, S', 1] where S' is the accumulated sequence length, E is the hidden size, and the first and last dimensions are of size 1.
skip
skip has the same input dimensions as the input.
The purpose of this input is to introduce skip (aka. residual) connections to previously computed tensors.
The skipLayerNormPlugin generates the following output:
output
output is a tensor with the same shape as the input.
Parameters
skipLayerNormPlugin has plugin creator class SkipLayerNormPluginDynamicCreator and plugin class CustomSkipLayerNormPluginDynamic.
The parameters are defined below and consists of the following attributes:
| Type | Parameter | Version | Description |
|---|---|---|---|
int |
type_id |
1, 2 | Integer encoding the DataType (0: FP32, 1: FP16, 2: INT8) |
int |
ld |
1 | The leading dimension of the input tensor, corresponding to the hidden size, denoted by E above. |
Weights |
beta |
1, 2, 3, 4 | The mean to normalize to. Shape: [1, 1, E] |
Weights |
gamma |
1, 2, 3, 4 | The standard deviation to normalize to. Shape: [1, 1, E] |
Weights |
bias |
1, 2 | An optional bias vector to add before normalization. Shape: [1, 1, E] |
Additional resources
License
For terms and conditions for use, reproduction, and distribution, see the TensorRT Software License Agreement documentation.
Changelog
February 2024 Add epsilon to avoid divide by zero.
October 2020 Add V2 plugin that supports variable sequence length. Add v3 plugin that supports int8 interleaved variable sequence length.
November 2019
This is the first release of this README.md file.
Known issues
This plugin only supports GPUs with compute capability >= 7.0. For more information see the CUDA GPU Compute Capability Support Matrix