!EiXx%SrSSKrSSKJr SSKJr SSKJr SSKr SSK J r SSKJ r JrJr /SQr\\\S 'S r\\ lS \ lS \S S4SjrSSSSSS.rS\4SjrS"S\4SjjrS"S\S\S S4Sjjr\R:S"S\\ \ -S\4Sjj5r\R:S\ 4Sj5rS \4Sjr SS K!J"r" \"RFr#\"RHr$\"RJr%\"RLr&\"RNr'\"RPr(\"RRr)\\%l\\&l\\'l\\(l\\)lSS!K!J*r*J+r+ g)#zvThis module contains functions and classes that alter the behavior of torch.nn.functional.scaled_dot_product_attentionN)Iterable)Union)warn) _SDPBackend)can_use_efficient_attentioncan_use_flash_attention SDPAParams) SDPBackend sdpa_kernelWARN_FOR_UNFUSED_KERNELSregister_flash_attention_implactivate_flash_attention_impllist_flash_attention_implscurrent_flash_attention_implrestore_flash_attention_impl__all__Fr paramsreturnc[(aN[U5(d[SSS9 [US5 [U5(d[SSS9 [US5 ggg)z If WARN_FOR_UNFUSED_KERNELS is set to True, this will raise warnings for all the reasons why the fused kernels can't be run. If using subclasses z*Efficient attention can't be used because:) stacklevelTz&Flash attention can't be used because:N)r rrr)rs X/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/nn/attention/__init__.py_raise_kernel_warningsr;sT  *622 =! L ' 5&v.. 9a H #FD 1/  CUDNN_ATTENTIONFLASH_ATTENTIONEFFICIENT_ATTENTIONMATH OVERRIDEABLE)cudnnflash mem_efficientmath overrideablenamec"[[U5$N)getattrr )r%s r_backend_from_stringr)Rs :t $$r with_priorityc>^/n[R5HNup#[[RSUS35"5(dM/UR [[ U55 MP U(a,[RR5m[UU4SjS9nU$)N_get_ _sdp_enabledc8>TR[U55$r')indexint)backend curr_prioritys r+_cur_sdpa_kernel_backends..^s-*=*=c'l*Kr)key) _backend_namesitemsr(torch_Cappendr _get_sdp_priority_ordersorted)r*backendsr%valr2s @r_cur_sdpa_kernel_backendsr?Vs{H#))+  588uTF,7 8 : : OOGJ4 5,88:  K  Orr= set_priorityc[R5H;up#[[U5U;n[[R SU35"U5 M= U(aUVs/sHn[ U5PM nn[R R5nUH$nXV;dM UR[ U55 M& [R RU5 ggs snf)N _set_sdp_use_) r6r7r(r r8r9r0r;r:_set_sdp_priority_order)r=r@r%r>enabledr1 user_priorityprevious_prioritys r _sdpa_kernelrGcs#))+ *c*h6M$01':,5=>X'WX >!HH<<>(G+$$S\2) ((7>sCc#X# [U[[45(d![S[ U5R 35e[U[5(aU/n[[ RU55n[US9n[X5 0v [X!5 g![X!5 f=f7f)a Context manager to select which backend to use for scaled dot product attention. .. warning:: This function is beta and subject to change. Args: backends (Union[List[SDPBackend], SDPBackend]): A backend or list of backends for scaled dot product attention. set_priority (bool=False): Whether the ordering of the backends is interpreted as their priority order. Example: .. code-block:: python from torch.nn.functional import scaled_dot_product_attention from torch.nn.attention import SDPBackend, sdpa_kernel # Only enable flash attention backend with sdpa_kernel(SDPBackend.FLASH_ATTENTION): scaled_dot_product_attention(...) # Enable the Math or Efficient attention backends with sdpa_kernel([SDPBackend.MATH, SDPBackend.EFFICIENT_ATTENTION]): scaled_dot_product_attention(...) # Enable the cuDNN or flash attention backends, and in that order with sdpa_kernel( [SDPBackend.CUDNN_ATTENTION, SDPBackend.FLASH_ATTENTION], set_priority=True ): scaled_dot_product_attention(...) This context manager can be used to select which backend to use for scaled dot product attention. Upon exiting the context manager, the previous state of the flags will be restored, enabling all backends. zQBackend must be an instance of SDPBackend or a list of SDPBackend instances, got )r*N) isinstancelistr AssertionErrortype__name__dictfromkeysr?rG)r=r@previous_backendss rr r qsF hz 2 3 3_`dem`n`w`w_x y  (J'':DMM(+,H1 M6X,&5 &5sAris|$. c &!! "  2: 2$ 2 * " %s% T  88 84 8D 8 16$z*Z716t1616j Z C  33## ) G G ) G G&AA(EE(EE+3(+3((0%*2'*2'r