Ein}SSKrSSKJr SSKJr SSKJr SSKJrJ r J r SSK r SSK r SSKJs Jr SSK JrJr SSKJrJrJr SSKJr SS KJrJr SS KJrJr SS KJ r SS K!J"r"J#r# SS K$J%r% /SQr&S\'\(\(4S\(S\(S\(S\'\(\(44 Sjr)S\'\(\(4S\(S\*\'\(\(44Sjr+S\(S\(S\ R$4Sjr,"SS\RZ5r."SS\RZ5r/"SS\RZ5r0"S S!\RZ5r1"S"S#\RZ5r2"S$S%\RZ5r3"S&S'\RZ5r4"S(S)\RZ5r5"S*S+\RZ5r6S=S,\(S-\*\(S.\*\(S/\7S0\(S1\(S2\ \S3\8S4\S\64S5jjr9"S6S7\5r:\"5\"S8\:Rv4S99SS:S;.S2\ \:S3\8S4\S\64S<jj55rN) OrderedDict)Sequence)partial)AnyCallableOptional)nnTensor)register_modelWeights WeightsEnum)_IMAGENET_CATEGORIES)_ovewrite_named_paramhandle_legacy_interface)Conv2dNormActivationSqueezeExcitation)StochasticDepth)ImageClassificationInterpolationMode)_log_api_usage_once)MaxVitMaxVit_T_Weightsmaxvit_t input_size kernel_sizestridepaddingreturncRUSU- SU--U-S-USU- SU--U-S-4$)Nr)rrrrs V/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torchvision/models/maxvit.py_get_conv_output_shaper$sJ A $q7{ 2v=A A $q7{ 2v=A n_blocksc/n[USSS5n[U5H"n[USSS5nURU5 M$ U$)zQUtil function to check that the input size is correct for a MaxVit configuration.r r!)r$rangeappend)rr&shapesblock_input_shape_s r#_make_block_input_shapesr.!sL F.z1aC 8_23DaAN '( Mr%heightwidthc[R"[R"[R"U5[R"U5/SS95n[R"US5nUSS2SS2S4USS2SSS24- nUR SSS5R 5nUSS2SS2S4==US- - ss'USS2SS2S4==US- - ss'USS2SS2S4==SU-S- -ss'URS5$)Nij)indexingr!r r)torchstackmeshgridarangeflattenpermute contiguoussum)r/r0coords coords_flatrelative_coordss r#_get_relative_position_indexr@+s [[f)=u||E?R(S^bc dF--*K!!Q*- AtQJ0GGO%--aA6AACOAq!G *Aq!G )Aq!GE A -   r ""r%c^\rSrSrSrSS\S\S\S\S\S\S \R4S \S \R4S \S S 4U4Sjjjr S\ S \ 4Sjr Sr U=r$)MBConv6aMBConv: Mobile Inverted Residual Bottleneck. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. expansion_ratio (float): Expansion ratio in the bottleneck. squeeze_ratio (float): Squeeze ratio in the SE Layer. stride (int): Stride of the depthwise convolution. activation_layer (Callable[..., nn.Module]): Activation function. norm_layer (Callable[..., nn.Module]): Normalization function. p_stochastic_dropout (float): Probability of stochastic depth. in_channels out_channelsexpansion_ratio squeeze_ratioractivation_layer. norm_layerp_stochastic_dropoutrNc >[TU]5 U US:g=(d X:gn U (aQ[R"XSSSS9/n US:Xa[R"SUSS9/U -n [R "U 6UlO[R"5Ul[X#-5n [X$-5n U(a[USS9Ul O[R"5Ul [5n U"U5U S '[UU SSS UUSS 9U S '[U U SUSUUU SS 9 U S'[X[RS9U S'[R"XSSS9U S'[R "U 5Ulg)Nr!T)rrbiasr r(rrrrowmodepre_normr)rrrrHrIinplaceconv_a)rrrrHrIgroupsrRconv_b) activationsqueeze_excitation)rDrErrLconv_c)super__init__r Conv2d AvgPool2d SequentialprojIdentityintrstochastic_depthrrrSiLUlayers)selfrDrErFrGrrHrIrJ should_projr^ mid_channels sqz_channels_layers __class__s r#rZMBConv.__init__Dsh  k@[%@ IIkQqW[\]D{ 61MNQUU t,DI DI<9: <78 $34Hu$UD !$&KKMD !-(5 0  -!  1  -!  ):,acahah(i$%II,ghostmmG, r%xclURU5nURURU55nX!-$)z Args: x (Tensor): Input tensor with expected layout of [B, C, H, W]. Returns: Tensor: Output tensor with expected layout of [B, C, H / stride, W / stride]. )r^rarcrdrkress r#forwardMBConv.forwards0iil  ! !$++a. 1wr%)rcr^ra))__name__ __module__ __qualname____firstlineno____doc__r`floatrr ModulerZr ro__static_attributes__ __classcell__ris@r#rBrB6s ,'*;-;-;- ;-  ;-  ;-#3 >2;-S"))^,;-$;- ;-;-z  F  r%rBct^\rSrSrSrS\S\S\SS4U4SjjrS\R4S jr S \S\4S jr S r U=r $) $RelativePositionalMultiHeadAttentionzRelative Positional Multi-Head Attention. Args: feat_dim (int): Number of input features. head_dim (int): Number of features per head. max_seq_len (int): Maximum sequence length. feat_dimhead_dim max_seq_lenrNcV>[TU]5 X-S:wa[SUSU35eX-UlX l[ [ R"U55UlX0l [R"XRUR-S-5Ul US-Ul [R"URUR-U5Ul[RR!["R$"SUR-S- SUR-S- -UR4["R&S95UlUR+S [-URUR55 ["RR.R1UR(S S 9 g) Nrz feat_dim: z must be divisible by head_dim: r(gr r!)dtyperelative_position_index{Gz?std)rYrZ ValueErrorn_headsrr`mathsqrtsizerr Linearto_qkv scale_factormerge parameter Parameterr5emptyfloat32relative_position_bias_tableregister_bufferr@init trunc_normal_)rdrrrris r#rZ-RelativePositionalMultiHeadAttention.__init__sK   ! #z(3ST\S]^_ _+   +./ &ii,,*F*JK $dNYYt}}t||;XF ,.LL,B,B KK!dii-!+DII 0ABDLLQY^YfYf g- ) 68TUYU^U^`d`i`i8jk  ##D$E$E4#Pr%cURRS5nURURURURS5nUR SSS5R 5nUR S5$)Nr4r rr!)rviewrrr:r; unsqueeze)rd bias_index relative_biass r#get_relative_positional_biasARelativePositionalMultiHeadAttention.get_relative_positional_biasss1166r: 99*EJJ4K[K[]a]m]moqr %--aA6AAC &&q))r%rkcURup#pEURURpvURU5n[R "USSS9upn U R X#XFU5RSSSSS5n U R X#XFU5RSSSSS5n U R X#XFU5RSSSSS5n XR-n [R"SX5n UR5n [R"X-SS9n [R"S X5nURSSSSS5R X#XE5nURU5nU$) z Args: x (Tensor): Input tensor with expected layout of [B, G, P, D]. Returns: Tensor: Output tensor with expected layout of [B, G, P, D]. r(r4)dimrr!r z!B G H I D, B G H J D -> B G H I Jz!B G H I J, B G H J D -> B G H I D)shaperrrr5chunkreshaper:reinsumrFsoftmaxr)rdrkBGPDHDHqkvqkvdot_prodpos_biasouts r#ro,RelativePositionalMultiHeadAttention.forwards8WW a dmm2kk!n++c1"-a IIaA" % - -aAq! < IIaA" % - -aAq! < IIaA" % - -aAq! < !! !<< CQJ44699X0b9ll>Lkk!Q1a(00q<jjo r%)rrrrrrrr) rrrsrtrurvr`rZr5r rroryrzr{s@r#r}r}s`QQQ Q  Q8*ell* Fr%r}cv^\rSrSrSrS\S\SS4U4SjjrS\RS\R4S jr S r U=r $) SwapAxeszPermute the axes of a tensor.abrNc:>[TU]5 XlX lgN)rYrZrr)rdrrris r#rZSwapAxes.__init__s r%rkc\[R"XRUR5nU$r)r5swapaxesrrrms r#roSwapAxes.forwardsnnQ/ r%)rr) rrrsrtrurvr`rZr5r roryrzr{s@r#rrs@'##$ %,,r%rcF^\rSrSrSrS U4SjjrS\S\S\4SjrSr U=r $) WindowPartitionz: Partition the input tensor into non-overlapping windows. rc">[TU]5 grrYrZrdris r#rZWindowPartition.__init__ r%rkpcURup4pVUnURX4XW-XvU-U5nURSSSSSS5nURX5U-Xg--Xw-U5nU$)z Args: x (Tensor): Input tensor with expected layout of [B, C, H, W]. p (int): Number of partitions. Returns: Tensor: Output tensor with expected layout of [B, H/P, W/P, P*P, C]. rr rr(r!rrr:)rdrkrrCrWrs r#roWindowPartition.forwardslWW a  IIaAFAAvq 1 IIaAq!Q ' IIaq&QV,aeQ 7r%r"rN rrrsrtrurvrZr r`roryrzr{s@r#rrs,CFr%rc N^\rSrSrSrS U4SjjrS\S\S\S\S\4 S jrS r U=r $) WindowDepartitionzg Departition the input tensor of non-overlapping windows into a feature volume of layout [B, C, H, W]. rc">[TU]5 grrrs r#rZWindowDepartition.__init__rr%rkr h_partitions w_partitionscURupVpxUn X4pURXZXX5nURSSSSSS5nURXXX-X-5nU$)a2 Args: x (Tensor): Input tensor with expected layout of [B, (H/P * W/P), P*P, C]. p (int): Number of partitions. h_partitions (int): Number of vertical partitions. w_partitions (int): Number of horizontal partitions. Returns: Tensor: Output tensor with expected layout of [B, C, H, W]. rrr!r(r rr) rdrkrrrrrPPrrHPWPs r#roWindowDepartition.forwards`gg b B IIaRA ) IIaAq!Q ' IIaBFBF +r%r"rrr{s@r#rrs;Cs#RXr%rc^\rSrSrSrS\S\S\S\S\\\4S\S \S \ R4S \S \ R4S \ S \ S\ SS4U4Sjjr S\ S\ 4SjrSrU=r$)PartitionAttentionLayeriav Layer for partitioning the input tensor into non-overlapping windows and applying attention to each window. Args: in_channels (int): Number of input channels. head_dim (int): Dimension of each attention head. partition_size (int): Size of the partitions. partition_type (str): Type of partitioning to use. Can be either "grid" or "window". grid_size (Tuple[int, int]): Size of the grid to partition the input tensor into. mlp_ratio (int): Ratio of the feature size expansion in the MLP layer. activation_layer (Callable[..., nn.Module]): Activation function to use. norm_layer (Callable[..., nn.Module]): Normalization function to use. attention_dropout (float): Dropout probability for the attention layer. mlp_dropout (float): Dropout probability for the MLP layer. p_stochastic_dropout (float): Probability of dropping out a partition. rDrpartition_sizepartition_type grid_size mlp_ratiorH.rIattention_dropout mlp_dropoutrJrNc >[T U]5 X-UlX lUSU-UlX@lXPlUS;a [S5eUS:XaX0RsUlUl OURUsUlUl [5Ul [5Ul US:Xa [SS5O[R "5UlUS:Xa [SS5O[R "5Ul[R&"U"U5[)XUS-5[R*"U 55Ul[R&"[R."U5[R0"XU-5U"5[R0"X-U5[R*"U 55Ul[5U S S 9Ulg) Nr)gridwindowz0partition_type must be either 'grid' or 'window'rrr rNrO)rYrZrr n_partitionsrrrrgr partition_oprdepartition_oprr r_partition_swapdepartition_swapr]r}Dropout attn_layer LayerNormr mlp_layerrstochastic_dropout) rdrDrrrrrrHrIrrrJris r#rZ PartitionAttentionLayer.__init__-su" ".  %aLN:," !3 3OP P X %+->->NDFDF!..NDFDF+-/12@F2Jhr2.PRP[P[P]4Bf4LR 0RTR]R]R_-- { # 1XYHY Z JJ( )   LL % IIk#: ;   IIk-{ ; JJ{ #  #22FU"Sr%rkcURSUR-URSUR-p2[R"URSUR-S:H=(a URSUR-S:HSR URUR55 UR XR5nUR U5nXRURU55-nXRURU55-nURU5nURXRX#5nU$)z Args: x (Tensor): Input tensor with expected layout of [B, C, H, W]. Returns: Tensor: Output tensor with expected layout of [B, C, H, W]. rr!z[Grid size must be divisible by partition size. Got grid size of {} and partition size of {}) rrr5_assertformatrrrrrrr)rdrkghgws r#roPartitionAttentionLayer.forwardgs"dff,dnnQ.?466.IB NN1  &! + Oq0ADFF0Ja0O i p p     a (    " ''(:; ; ''q(9: :  ! !! $   662 2r%)rrrrrrrrrrrrrr)rrrsrtrurvr`strtuplerr rxrwrZr roryrzr{s@r#rrs"8T8T8T  8T  8Tc?8T8T#3 >28TS"))^,8T!8T8T$8T !8TtFr%rc^\rSrSrSrS\S\S\S\S\S\S \R4S \S \R4S \S \S \S\S\S\S\ \\4SS4U4Sjjr S\ S\ 4Sjr SrU=r$) MaxVitLayeria MaxVit layer consisting of a MBConv layer followed by a PartitionAttentionLayer with `window` and a PartitionAttentionLayer with `grid`. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. expansion_ratio (float): Expansion ratio in the bottleneck. squeeze_ratio (float): Squeeze ratio in the SE Layer. stride (int): Stride of the depthwise convolution. activation_layer (Callable[..., nn.Module]): Activation function. norm_layer (Callable[..., nn.Module]): Normalization function. head_dim (int): Dimension of the attention heads. mlp_ratio (int): Ratio of the MLP layer. mlp_dropout (float): Dropout probability for the MLP layer. attention_dropout (float): Dropout probability for the attention layer. p_stochastic_dropout (float): Probability of stochastic depth. partition_size (int): Size of the partitions. grid_size (Tuple[int, int]): Size of the input feature grid. rDrErGrFrrI.rHrrrrrJrrrNc">[TU]5 [5n[UUUUUUUU S9US'[ UUU SUU U[ R U U U S9 US'[ UUU SUU U[ R U U U S9 US'[ R"U5Ulg)N)rDrErFrGrrHrIrJMBconvr) rDrrrrrrHrIrrrJwindow_attentionrgrid_attention) rYrZrrBrr rr]rc)rdrDrErGrFrrIrHrrrrrJrrrcris r#rZMaxVitLayer.__init__s* )m"#%+'-!!5  x&=$)#-||/#!5 & !"$;$)!-||/#!5 $  mmF+ r%rkc(URU5nU$zu Args: x (Tensor): Input tensor of shape (B, C, H, W). Returns: Tensor: Output tensor of shape (B, C, H, W). rc)rdrks r#roMaxVitLayer.forwards KKNr%r )rrrsrtrurvr`rwrr rxrrZr roryrzr{s@r#rrs(?,?, ?,  ?,  ?,?,S"))^,?,#3 >2?,?,?,?,!?, $!?,$%?,&c?'?,( )?,BFr%rc^\rSrSrSrS\S\S\S\S\S\R4S \S\R4S \S \S \S \S\S\ \\4S\S\ \SS4U4Sjjr S\ S\ 4SjrSrU=r$) MaxVitBlockia A MaxVit block consisting of `n_layers` MaxVit layers. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. expansion_ratio (float): Expansion ratio in the bottleneck. squeeze_ratio (float): Squeeze ratio in the SE Layer. activation_layer (Callable[..., nn.Module]): Activation function. norm_layer (Callable[..., nn.Module]): Normalization function. head_dim (int): Dimension of the attention heads. mlp_ratio (int): Ratio of the MLP layer. mlp_dropout (float): Dropout probability for the MLP layer. attention_dropout (float): Dropout probability for the attention layer. p_stochastic_dropout (float): Probability of stochastic depth. partition_size (int): Size of the partitions. input_grid_size (Tuple[int, int]): Size of the input feature grid. n_layers (int): Number of layers in the block. p_stochastic (List[float]): List of probabilities for stochastic depth for each layer. rDrErGrFrI.rHrrrrrinput_grid_sizen_layers p_stochasticrNcp>[TU]5 [U5U :Xd[SU SUS35e[R "5Ul[U SSSS9Ul[U5HLunnUS:XaSOSnU=R [US:XaUOUUUUUUUUUU U U URUS 9/- slMN g) Nz'p_stochastic must have length n_layers=z, got p_stochastic=.r(r r!rMr)rDrErGrFrrIrHrrrrrrrJ) rYrZlenrr ModuleListrcr$r enumerater)rdrDrErGrFrIrHrrrrrrrridxrrris r#rZMaxVitBlock.__init__s, < H,FxjPcdpcqqrst tmmo /QWXbcd -FC(QF KK/2ax \!-"/$3!)%5%' +&7#1"nn)* K.r%rkc<URH nU"U5nM U$r r )rdrklayers r#roMaxVitBlock.forward-s [[EaA!r%)rrc)rrrsrtrurvr`rwrr rxrlistrZr roryrzr{s@r#r r s*11 1  1  1S"))^,1#3 >21111!1 !1"sCx#1&'1(5k)1* +1f  F  r%r c!^\rSrSrSrS\R SSSSSS4S\\\4S \S \S \ \S \ \S \S\ S\ \ S\R4S\ S\R4S\ S\ S\S\ S\ S\SS4 U4SjjjrS\S\4SjrSrSrU=r$)ri9a1 Implements MaxVit Transformer from the `MaxViT: Multi-Axis Vision Transformer `_ paper. Args: input_size (Tuple[int, int]): Size of the input image. stem_channels (int): Number of channels in the stem. partition_size (int): Size of the partitions. block_channels (List[int]): Number of channels in each block. block_layers (List[int]): Number of layers in each block. stochastic_depth_prob (float): Probability of stochastic depth. Expands to a list of probabilities for each layer that scales linearly to the specified value. squeeze_ratio (float): Squeeze ratio in the SE Layer. Default: 0.25. expansion_ratio (float): Expansion ratio in the MBConv bottleneck. Default: 4. norm_layer (Callable[..., nn.Module]): Normalization function. Default: None (setting to None will produce a `BatchNorm2d(eps=1e-3, momentum=0.01)`). activation_layer (Callable[..., nn.Module]): Activation function Default: nn.GELU. head_dim (int): Dimension of the attention heads. mlp_ratio (int): Expansion ratio of the MLP layer. Default: 4. mlp_dropout (float): Dropout probability for the MLP layer. Default: 0.0. attention_dropout (float): Dropout probability for the attention layer. Default: 0.0. num_classes (int): Number of classes. Default: 1000. Ng?rrqir stem_channelsrblock_channels block_layersrstochastic_depth_probrI.rHrGrFrrr num_classesrc>[TU]5 [U5 SnUc[[R SSS9n[ U[U55n[U5H6unnUSU-S:wdUSU-S:wdM [SUSUS US US 3 5e [R"[UUSS UU S SS9[X"SSSSSS95Ul [USS SS9nX0l[R"5UlU/USS-nUn["R$"SU['U55R)5nSn[+UUU5HZunnnUR R-[/UUU U UU UU U UUUUUUUU-S95 UR SR0nUU- nM\ [R"[R2"S5[R4"5[R6"US5[R8"USUS5[R:"5[R8"USUS S95UlUR?5 g)Nr(gMbP?g{Gz?)epsmomentumrr!z Input size z of block z$ is not divisible by partition size zx. Consider changing the partition size or the input size. Current configuration yields the following block input sizes: rr F)rrIrHrLrRT)rrIrHrLrMr4)rDrErGrFrIrHrrrrrrrr)rL) rYrZrrr BatchNorm2dr.rrrr]rstemr$rrblocksnplinspacer<tolistzipr*r rAdaptiveAvgPool2dFlattenrrTanh classifier _init_weights)rdrrrrrrr rIrHrGrFrrrr!input_channelsblock_input_sizesrblock_input_sizerDrErp_idx in_channel out_channel num_layersris r#rZMaxVit.__init__Ns: D!   TDIJ 5Z^ATU%./@%A !C!"^3q857WW#!" #&7t#s(Ot  ttS t3itt %t"Xc299n56#t$#3 >2%t()t*+t./t01t2!3t67t8 9ttlF + +r%rrrrr rrweightsprogresskwargsc dUbe[US[URS55 URSSURSS:Xde[USURS5 URSS5n [ S UUUUUUU S.UD6n UbU R UR US S 95 U $) Nr! categoriesmin_sizerr!rrM)rrrr rrrT)rG check_hashr")rrmetapoprload_state_dictget_state_dict) rrrr rrrFrGrHrmodels r#_maxvitrTs$fmSl9S5TU||J'*gll:.Fq.IIIIflGLL4LML*5J  #%!3%   E g44hSW4XY Lr%c f\rSrSr\"S\"\SS\RS9\ SSSSS S S .0S S SS.S9r \ r Sr g)riz9https://download.pytorch.org/models/maxvit_t-bc5ab103.pthrM) crop_size resize_size interpolationirLzLhttps://github.com/pytorch/vision/tree/main/references/classification#maxvitz ImageNet-1KgT@g|?5.X@)zacc@1zacc@5gZd;@gK7]@zThese weights reproduce closely the results of the paper using a similar training recipe. They were trained with a BatchNorm2D momentum of 0.99 instead of the more correct 0.01.)rJ num_paramsrKrecipe_metrics_ops _file_size_docs)url transformsrOr"N) rrrsrtrur rrrBICUBICr IMAGENET1K_V1DEFAULTryr"r%r#rrsb G 3CO`OhOh /""d##  !g M.Gr%r pretrained)rFT)rFrGc \[RU5n[SS/SQ/SQSSSUUS.UD6$) aF Constructs a maxvit_t architecture from `MaxViT: Multi-Axis Vision Transformer `_. Args: weights (:class:`~torchvision.models.MaxVit_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.MaxVit_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.maxvit.MaxVit`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.MaxVit_T_Weights :members: @)rfi)r r rr g?)rrrrr rrFrGr")rverifyrT)rFrGrHs r#rrsH.%%g.G  *!!    r%)NF)=r collectionsrcollections.abcr functoolsrtypingrrrnumpyr(r5torch.nn.functionalr functionalrr torchvision.models._apir r r torchvision.models._metartorchvision.models._utilsrrtorchvision.ops.miscrr torchvision.ops.stochastic_depthrtorchvision.transforms._presetsrrtorchvision.utilsr__all__rr`r$rr.r@rxrBr}rrrrrr rrwboolrTrrbrr"r%r#r|s[ #$** HH9TH<R1 uS#XSRU`chmnqsvnvhwsCxCDQVWZ\_W_Q`La##S#U\\#TRYYTnF299FR ryy bii4