paddlenlp/ops/faster_transformer/transformer/encoder.py
def encoder_forward(self, src, src_mask=None, cache=None):
    """
    Redefines `forward` function of `paddle.nn.TransformerEncoder` for
    integrating FasterTransformer for inference.

    The original `forward` function would not be replaced unless
    `enable_faster_encoder` is called by objects of its base class. After
    replacing, objects of `paddle.nn.TransformerEncoder` also have the same
    member variables as before.

    After inference, `disable_faster_encoder` could be called to restore the
    `forward` function of `paddle.nn.TransformerEncoder` and
    `paddle.nn.TransformerEncoderLayer`.

    Args:
        src (Tensor):
            The input of Transformer encoder. It is a tensor
            with shape `[batch_size, sequence_length, d_model]`. The data
            type should be float32 or float16.
        src_mask (Tensor, optional):
            A tensor used in multi-head attention to prevents attention to
            some unwanted positions, usually the paddings or the subsequent
            positions. It is a tensor with shape `[batch_size, 1, 1, sequence_length]`.
            The data type must be float, the unwanted positions have `-INF` values or other non-zeros
            and the wanted positions must be 0.0.
    Returns:
        output (Tensor|tuple):
            It is a tensor that has the same shape and data type as `src`,
            representing the output of Transformer encoder. Or a tuple if
            `cache` is not None, except for encoder output, the tuple includes
            the new cache which is same as input `cache` argument but
            `incremental_cache` in it has an incremental length. See
            `paddle.nn.MultiHeadAttention.gen_cache` and
            `paddle.nn.MultiHeadAttention.forward` for more details.
    """

    if src_mask.dtype == paddle.float16:
        src_mask = paddle.cast(src_mask, "float32")

    src_mask = src_mask == 0.0
    src_mask = paddle.cast(src_mask, src.dtype)

    # transpose_src_mask: [batch_size, 1, sequence_length, 1]
    transpose_src_mask = paddle.transpose(src_mask, perm=[0, 1, 3, 2])

    # src_mask: [batch_size, 1, sequence_length, sequence_length]
    src_mask = src_mask * transpose_src_mask
    output = src
    for i, layer in enumerate(self.layers):
        output = layer(output, src_mask)
    if self.norm is not None:
        output = self.norm(output)
    return output

    Redefines `forward` function of `paddle.nn.TransformerEncoder` for
    integrating FasterTransformer for inference.

    The original `forward` function would not be replaced unless
    `enable_faster_encoder` is called by objects of its base class. After
    replacing, objects of `paddle.nn.TransformerEncoder` also have the same
    member variables as before.

    After inference, `disable_faster_encoder` could be called to restore the
    `forward` function of `paddle.nn.TransformerEncoder` and
    `paddle.nn.TransformerEncoderLayer`.

    Args:
        src (Tensor):
            The input of Transformer encoder. It is a tensor
            with shape `[batch_size, sequence_length, d_model]`. The data
            type should be float32 or float16.
        src_mask (Tensor, optional):
            A tensor used in multi-head attention to prevents attention to
            some unwanted positions, usually the paddings or the subsequent
            positions. It is a tensor with shape `[batch_size, 1, 1, sequence_length]`.
            The data type must be float, the unwanted positions have `-INF` values or other non-zeros
            and the wanted positions must be 0.0.
    Returns:
        output (Tensor|tuple):
            It is a tensor that has the same shape and data type as `src`,
            representing the output of Transformer encoder. Or a tuple if
            `cache` is not None, except for encoder output, the tuple includes
            the new cache which is same as input `cache` argument but
            `incremental_cache` in it has an incremental length. See
            `paddle.nn.MultiHeadAttention.gen_cache` and
            `paddle.nn.MultiHeadAttention.forward` for more details.
    
Redefines `forward` function of `paddle.nn.TransformerEncoder` for
integrating FasterTransformer for inference.

The original `forward` function would not be replaced unless
`enable_faster_encoder` is called by objects of its base class. After
replacing, objects of `paddle.nn.TransformerEncoder` also have the same
member variables as before.


Redefines ` forward ` function of ` paddle . nn . TransformerEncoder ` for integrating FasterTransformer for inference . The original ` forward ` function would not be replaced unless ` enable_faster_encoder ` is called by objects of its base class . After replacing objects of ` paddle . nn . TransformerEncoder ` also have the same member variables as before .
def encoder_forward(self, src, src_mask=None, cache=None):
    if src_mask.dtype == paddle.float16:
        src_mask = paddle.cast(src_mask, "float32")
    src_mask = src_mask == 0.0
    src_mask = paddle.cast(src_mask, src.dtype)
    transpose_src_mask = paddle.transpose(src_mask, perm=[0, 1, 3, 2])
    src_mask = src_mask * transpose_src_mask
    output = src
    for i, layer in enumerate(self.layers):
        output = layer(output, src_mask)
    if self.norm is not None:
        output = self.norm(output)
    return output
def encoder_forward ( self , src , src_mask = None , cache = None ) : if src_mask . dtype == paddle . float16 : src_mask = paddle . cast ( src_mask , "float32" ) src_mask = src_mask == 0.0 src_mask = paddle . cast ( src_mask , src . dtype ) transpose_src_mask = paddle . transpose ( src_mask , perm = [ 0 , 1 , 3 , 2 ] ) src_mask = src_mask * transpose_src_mask output = src for i , layer in enumerate ( self . layers ) : output = layer ( output , src_mask ) if self . norm is not None : output = self . norm ( output ) return output
Redefines `forward` function of `paddle.nn.TransformerEncoder` for
integrating FasterTransformer for inference.
Redefines ` forward ` function of ` paddle . nn . TransformerEncoder ` for integrating FasterTransformer for inference .
"""
    Redefines `forward` function of `paddle.nn.TransformerEncoder` for
    integrating FasterTransformer for inference.

    The original `forward` function would not be replaced unless
    `enable_faster_encoder` is called by objects of its base class. After
    replacing, objects of `paddle.nn.TransformerEncoder` also have the same
    member variables as before.

    After inference, `disable_faster_encoder` could be called to restore the
    `forward` function of `paddle.nn.TransformerEncoder` and
    `paddle.nn.TransformerEncoderLayer`.

    Args:
        src (Tensor):
            The input of Transformer encoder. It is a tensor
            with shape `[batch_size, sequence_length, d_model]`. The data
            type should be float32 or float16.
        src_mask (Tensor, optional):
            A tensor used in multi-head attention to prevents attention to
            some unwanted positions, usually the paddings or the subsequent
            positions. It is a tensor with shape `[batch_size, 1, 1, sequence_length]`.
            The data type must be float, the unwanted positions have `-INF` values or other non-zeros
            and the wanted positions must be 0.0.
    Returns:
        output (Tensor|tuple):
            It is a tensor that has the same shape and data type as `src`,
            representing the output of Transformer encoder. Or a tuple if
            `cache` is not None, except for encoder output, the tuple includes
            the new cache which is same as input `cache` argument but
            `incremental_cache` in it has an incremental length. See
            `paddle.nn.MultiHeadAttention.gen_cache` and
            `paddle.nn.MultiHeadAttention.forward` for more details.
    """ # transpose_src_mask: [batch_size, 1, sequence_length, 1] # src_mask: [batch_size, 1, sequence_length, sequence_length]
output (Tensor|tuple):
It is a tensor that has the same shape and data type as `src`,
representing the output of Transformer encoder. Or a tuple if
`cache` is not None, except for encoder output, the tuple includes
the new cache which is same as input `cache` argument but
`incremental_cache` in it has an incremental length. See
`paddle.nn.MultiHeadAttention.gen_cache` and
`paddle.nn.MultiHeadAttention.forward` for more details.
output ( Tensor|tuple ) : It is a tensor that has the same shape and data type as ` src ` representing the output of Transformer encoder . Or a tuple if ` cache ` is not None except for encoder output the tuple includes the new cache which is same as input ` cache ` argument but ` incremental_cache ` in it has an incremental length . See ` paddle . nn . MultiHeadAttention . gen_cache ` and ` paddle . nn . MultiHeadAttention . forward ` for more details .
self
src
The input of Transformer encoder. It is a tensor
with shape `[batch_size, sequence_length, d_model]`. The data
type should be float32 or float16.
The input of Transformer encoder . It is a tensor with shape ` [ batch_size sequence_length d_model ] ` . The data type should be float32 or float16 .
src_mask
A tensor used in multi-head attention to prevents attention to
some unwanted positions, usually the paddings or the subsequent
positions. It is a tensor with shape `[batch_size, 1, 1, sequence_length]`.
The data type must be float, the unwanted positions have `-INF` values or other non-zeros
and the wanted positions must be 0.0.
A tensor used in multi - head attention to prevents attention to some unwanted positions usually the paddings or the subsequent positions . It is a tensor with shape ` [ batch_size 1 1 sequence_length ] ` . The data type must be float the unwanted positions have ` - INF ` values or other non - zeros and the wanted positions must be 0 . 0 .
cache