paddlenlp/transformers/albert/tokenizer.py
def build_offset_mapping_with_special_tokens(self,
                                                 offset_mapping_0,
                                                 offset_mapping_1=None):
        """
        Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.

        A Albert offset_mapping has the following format:

        - single sequence:      ``(0,0) X (0,0)``
        - pair of sequences:        ``(0,0) A (0,0) B (0,0)``

        Args:
            offset_mapping_ids_0 (List[tuple]):
                List of wordpiece offsets to which the special tokens will be added.
            offset_mapping_ids_1 (List[tuple], optional):
                Optional second list of wordpiece offsets for offset mapping pairs. Defaults to None.

        Returns:
            List[tuple]: A list of wordpiece offsets with the appropriate offsets of special tokens.
        """
        return self.tokenizer.build_offset_mapping_with_special_tokens(
            offset_mapping_0, offset_mapping_1=offset_mapping_1)

        Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.

        A Albert offset_mapping has the following format:

        - single sequence:      ``(0,0) X (0,0)``
        - pair of sequences:        ``(0,0) A (0,0) B (0,0)``

        Args:
            offset_mapping_ids_0 (List[tuple]):
                List of wordpiece offsets to which the special tokens will be added.
            offset_mapping_ids_1 (List[tuple], optional):
                Optional second list of wordpiece offsets for offset mapping pairs. Defaults to None.

        Returns:
            List[tuple]: A list of wordpiece offsets with the appropriate offsets of special tokens.
        
Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.
A Albert offset_mapping has the following format.


Build offset map from a pair of offset map by concatenating and adding offsets of special tokens . A Albert offset_mapping has the following format .
def build_offset_mapping_with_special_tokens(self,
                                                 offset_mapping_0,
                                                 offset_mapping_1=None):
        return self.tokenizer.build_offset_mapping_with_special_tokens(
            offset_mapping_0, offset_mapping_1=offset_mapping_1)
def build_offset_mapping_with_special_tokens ( self , offset_mapping_0 , offset_mapping_1 = None ) : return self . tokenizer . build_offset_mapping_with_special_tokens ( offset_mapping_0 , offset_mapping_1 = offset_mapping_1 )
Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.
Build offset map from a pair of offset map by concatenating and adding offsets of special tokens .
"""
        Build offset map from a pair of offset map by concatenating and adding offsets of special tokens.

        A Albert offset_mapping has the following format:

        - single sequence:      ``(0,0) X (0,0)``
        - pair of sequences:        ``(0,0) A (0,0) B (0,0)``

        Args:
            offset_mapping_ids_0 (List[tuple]):
                List of wordpiece offsets to which the special tokens will be added.
            offset_mapping_ids_1 (List[tuple], optional):
                Optional second list of wordpiece offsets for offset mapping pairs. Defaults to None.

        Returns:
            List[tuple]: A list of wordpiece offsets with the appropriate offsets of special tokens.
        """
A list of wordpiece offsets with the appropriate offsets of special tokens.
A list of wordpiece offsets with the appropriate offsets of special tokens .
List[tuple]
self
offset_mapping_0
offset_mapping_1
offset_mapping_ids_0
List of wordpiece offsets to which the special tokens will be added.
List of wordpiece offsets to which the special tokens will be added .
offset_mapping_ids_1
Optional second list of wordpiece offsets for offset mapping pairs. Defaults to None.
Optional second list of wordpiece offsets for offset mapping pairs . Defaults to None .
None