qyij0JSrSSKJr SSKrSSKJr SSKJr SSKJ r SSK J r SS K J r Jr \ (a SS KJr S S KJr 0r\\"S S9"SS555r"SS5r\"S5rSSjrSSjrSSSjjr\R4"5rSSjrSSS.Sjjrg) z Contains the logic for automatic additional output capture with our forward decorators. This mostly describe the hooks used and the logic to make capture thread/context safe. ) annotationsN) ContextVar) dataclasswraps) TYPE_CHECKING)is_torchdynamo_compilingrequires)nnPreTrainedModel)torch)backendscN\rSrSr%SrS\S'SrS\S'SrS \S 'SrS \S 'S r g) OutputRecorder'a Configuration for recording outputs from a model via hooks. Attributes: target_class (Type): The class (e.g., nn.Module) to which the hook will be attached. index (Optional[int]): If the output is a tuple/list, optionally record only at a specific index. layer_name (Optional[str]): Name of the submodule to target (if needed), e.g., "transformer.layer.3.attn". class_name (Optional[str]): Name of the class to which the hook will be attached. Could be the suffix of class name in some cases. ztype[nn.Module] target_classrintindexN str | None layer_name class_name) __name__ __module__ __qualname____firstlineno____doc____annotations__rrr__static_attributes__r`/home/mande/repo/quber/.venv/lib/python3.13/site-packages/transformers/utils/output_capturing.pyrr's,"!E3N!J !!J !r#rc0\rSrSrSrSrSrSrSrSr g) CompileableContextVar:a Convenience wrapper around a ContextVar for usage with `torch.compile`. This behaves exactly as a `ContextVar`, except when compilation is triggered in which case it behaves as a simple global variable. This is useful as `torch.compile` cannot trace the `get` method of `ContextVar`. This however means that the access to the underlying variable is not thread-safe when compilation is triggered. c>[USS9UlSUlSUlg)N)defaultF)r context_var global_var compiling)selfnames r$__init__CompileableContextVar.__init__Bs%dD9r#cpUR(a UR$URR5$N)r,r+r*get)r-s r$r3CompileableContextVar.getGs( >>?? "##'') )r#cr[5(aXlSUlgURR U5$)NT)r r+r,r*set)r-values r$r6CompileableContextVar.setNs1 # % %#O!DN##''. .r#cUR(dUcSUlSUlgURRU5 g)NF)r,r+r*reset)r-tokens r$r:CompileableContextVar.resetVs/ >>U]"DO"DN    " "5 )r#)r,r*r+N) rrrrr r/r3r6r:r"rr#r$r&r&:s */*r#r&output_collectorc8^^UU4SjnURU5 g)zaInstall the forward hook needed to capture the output described by `key` and `index` in `module`.cL>[R5nUbTUR5;agTS:Xa)[UT5S:XaUTR US5 [ U[ 5(dUTR U5 gUTbUTR UT5 gg)N hidden_statesr)_active_collectorr3keyslenappend isinstancetuple)moduleargsoutputcollected_outputsrkeys r$output_capturing_hook.output_capturing_hookes-113  $3D3I3I3K(K  / !c*;C*@&AQ&F c " ) )$q' 2&%(( c " ) )& 1 E] & c " ) )&- 8'r#N)register_forward_hook)rGrKrrLs `` r$install_output_capturing_hookrObs 9   !67r#cSSKJn UR5H6upE[XS5(d[ XQSU3U5 M([ XQSU3S9 M8 UHupgUR b[XR 5(d1URcM;URUR5(dM]URbURU;aM|[XUR5 M g)a Recursively install all output capturing hooks on all submodules of `parent_module`. Note that we need to use this recursive approach instead of simply iterating over all modules, because we want to respect the `capture_tasks` of all individual submodels (`PreTrainedModel` instances) in the graph. That is, once we reach a submodel in the graph, its children should use this submodel's `capture_tasks`, but other parts of the graph should not. r r.)prefixN) modeling_utilsrnamed_childrenrErecursively_install_hooks"install_all_output_capturing_hooksrrendswithrrOr) parent_module module_name capture_tasksrr.rGrKspecss r$rUrUvs1&446 &22 %f Qtf.E} U /v Qtf>U V 7$     *z-I[I[/\/\    ([-A-A%BRBR-S-S+0@0@ 0S )-ekk J$r#c[R[UR55=(d 0n/nUR 5HupE[ U[ 5(dU/nUHpn[ U[5(dFSU;aSOSn[ U[5(dSOUn[ U[5(dUOSn [XUS9nURXF45 Mr M UbUOSn[XU5 [USS5 g) z Install the output recording hooks on all the modules in `model`. This will take care of correctly dispatching the `_can_record_outputs` property of each individual submodels in case of composite models. r@rr N)rrr!_output_capturing_hooks_installedT) _CAN_RECORD_REGISTRYr3str __class__itemsrElistrrDrUsetattr) modelrR capture_flagsrZrK layer_specsr[rrrs r$rVrVs ),,S-ABHbMM)//1+t,,&-K Ee^44,3)3E3)?)?TU ,6uc,B,Bu &LZde  # . !2)VrFe]; E6=r#c[USS5(ag[ [USS5(a SSS5 g[U5 SSS5 g!,(df  g=f)z Check if the model already has output capturing hooks installed, and install them if it is not already the case. Note that this is thread-safe, in case 2 (or more) threads want to install them concurrently. r^FN)getattr_hook_installation_lockrV)res r$maybe_install_capturing_hooksrksMu95AA  5=u E E  ! +51 ! sA  A  AT)tie_last_hidden_statesc*^U4SjnUbU"U5$U$)a Decorator to intercept specific layer outputs through hooks. The hooks are installed only once and lazily, the first time output capture is requested with the `output_xxx` kwargs/config. The implementation is fully context/thread safe, except when using `torch.compile`, as dynamo is unable to trace through `ContextVar` methods. Args: tie_last_hidden_states (`bool`, *optional*, defaults to `True`): Whether to overwrite `out.hidden_states[-1]` with the `out.last_hidden_state`. This is true for all language models and should be toggled off only if `out.hidden_states[-1]` has to be the hidden state before last layer norm, which is needed for some vision models (e.g. CLIP, SigLIP) c4>^[T5UU4Sj5nU$)Nc >URS[URSS55n[R [ UR 55=(d 0nUVs0sH4nSU3UR SU3[URSU3S55_M6 nnSU;a*UR S[URSS55US'SU;a*UR S[URSS55US 'UR5VVs0sH!upWU(dMURSS 5/_M# nnn[U5S :a [U5 [RU5n T "U/UQ70UD6n [RU 5 UGHn U S :XaT (dOo[U S 5(a'XSSX'XRU R 5 O7[U S5(a&XSSX'XRU R"5 [%X5X'MU S:Xaj['XK[(5(aA[XK5S:Xa0[%XS SS25X'[%XSSS25U SU -'M[%X5X'GM[%X5X'GM USLaU R+5n U $s snfs snnf![RU 5 f=f)N return_dictToutput_Fcross_attentionsoutput_attentionsoutput_cross_attentionsmask_decoder_attentionsoutput_mask_decoder_attentionsr]rr@vision_hidden_stateslast_hidden_state attentionsr r cross_)popriconfigr_r3r`rarbreplacerCrkrAr6r:hasattrrDrwryrFrErcto_tuple)r-rHkwargsrpcapturable_flagskrecordable_keysvrJ output_tokenoutputsrKfuncrls r$wrapper4capture_outputs..wrapped_fn..wrappers!**]GDKKX\4]^K 477DNN8KLRPR *)A! vzzGA3-PWXYWZm]b9cdd)  "%55=CZZ'>QSX)Y> 9:),<QSX)YE @AKZJ_J_Ja gJa$!ef!=9b!92!=Ja  g$%)-d3,001BCL 6t5d5f5"'' 5)/)1 *@AA1B1G1L).).55g6R6RS *=>>1B1G1L).).55g6O6OP#():)?#@GLL(!"2"7>>3GWG\C]abCb',->-CADqD-I'J 278I8NqtRSt8T2U3/',->-C'D #():)?#@GL)),e#!**,No!h"'' 5s;J9 J>J>) KKr)rrrls` r$ wrapped_fn#capture_outputs..wrapped_fns! t=  = ~r#r)rrlrs ` r$capture_outputsrs#AF $ r#)rG nn.ModulerKr`rrreturnNone)rXrrYr`rZz list[tuple[str, OutputRecorder]]rrr2)rerrRrrr)rerrr)r __future__r threading contextvarsr dataclassesr functoolsrtypingr import_utilsr r rr rSrr_rr&rArOrUrVLockrjrkrrr#r$rs #"! <0  :"" ""!*!*J**<=8(KK+.K?_K K@>:$..*2&TTr#