Shortcuts

# PCPVT¶

class mmpretrain.models.backbones.PCPVT(arch, in_channels=3, out_indices=(3,), qkv_bias=False, drop_rate=0.0, attn_drop_rate=0.0, drop_path_rate=0.0, norm_cfg={'type': 'LN'}, norm_after_stage=False, init_cfg=None)[源代码]

The backbone of Twins-PCPVT.

This backbone is the implementation of Twins: Revisiting the Design of Spatial Attention in Vision Transformers.

• arch (dict, str) –

PCPVT architecture, a str value in arch zoo or a detailed configuration dict with 7 keys, and the length of all the values in dict should be the same:

• depths (List[int]): The number of encoder layers in each stage.

• embed_dims (List[int]): Embedding dimension in each stage.

• patch_sizes (List[int]): The patch sizes in each stage.

• strides (List[int]): The strides in each stage.

• mlp_ratios (List[int]): The ratios of mlp in each stage.

• sr_ratios (List[int]): The ratios of GSA-encoder layers in each stage.

• in_channels (int) – Number of input channels. Defaults to 3.

• out_indices (tuple[int]) – Output from which stages. Defaults to (3, ).

• qkv_bias (bool) – Enable bias for qkv if True. Defaults to False.

• drop_rate (float) – Probability of an element to be zeroed. Defaults to 0.

• attn_drop_rate (float) – The drop out rate for attention layer. Defaults to 0.0

• drop_path_rate (float) – Stochastic depth rate. Defaults to 0.0.

• norm_cfg (dict) – Config dict for normalization layer. Defaults to dict(type='LN').

• norm_after_stage (bool, List[bool]) – Add extra norm after each stage. Defaults to False.

• init_cfg (dict, optional) – The Config for initialization. Defaults to None.

>>> from mmpretrain.models import PCPVT
>>> import torch
>>> pcpvt_cfg = {'arch': "small",
>>>              'norm_after_stage': [False, False, False, True]}
>>> model = PCPVT(**pcpvt_cfg)
>>> x = torch.rand(1, 3, 224, 224)
>>> outputs = model(x)
>>> print(outputs[-1].shape)
torch.Size([1, 512, 7, 7])
>>> pcpvt_cfg['norm_after_stage'] = [True, True, True, True]
>>> pcpvt_cfg['out_indices'] = (0, 1, 2, 3)
>>> model = PCPVT(**pcpvt_cfg)
>>> outputs = model(x)
>>> for feat in outputs:
>>>     print(feat.shape)
torch.Size([1, 64, 56, 56])
torch.Size([1, 128, 28, 28])
torch.Size([1, 320, 14, 14])
torch.Size([1, 512, 7, 7])


© Copyright 2020, OpenMMLab. Revision bb59c9ad.

Built with Sphinx using a theme provided by Read the Docs.
{"ft_lang_label":"__label__en","ft_lang_prob":0.60279924,"math_prob":0.92940617,"size":2231,"snap":"2023-40-2023-50","text_gpt3_token_len":656,"char_repetition_ratio":0.13560843,"word_repetition_ratio":0.012820513,"special_character_ratio":0.32451814,"punctuation_ratio":0.2293578,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.96989757,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2023-09-21T19:57:27Z\",\"WARC-Record-ID\":\"<urn:uuid:384e24a6-4814-42fe-8fae-176269347afb>\",\"Content-Length\":\"53307\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:6d8fcc39-f6ab-4c69-a19b-44d4fa8f815b>\",\"WARC-Concurrent-To\":\"<urn:uuid:5514aa7d-562d-4f3f-ba98-fe637a09fe26>\",\"WARC-IP-Address\":\"104.17.32.82\",\"WARC-Target-URI\":\"https://mmpretrain.readthedocs.io/zh_CN/latest/api/generated/mmpretrain.models.backbones.PCPVT.html\",\"WARC-Payload-Digest\":\"sha1:CABWIRTTTWKNJSWXI4UPO6MKVASQZ2QK\",\"WARC-Block-Digest\":\"sha1:XCXZ3WNCVPV5NI5DMZ4R6XDV7R6KST2H\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2023/CC-MAIN-2023-40/CC-MAIN-2023-40_segments_1695233506029.42_warc_CC-MAIN-20230921174008-20230921204008-00365.warc.gz\"}"}