EiU52SSKrSSKrSSKJr SSKJrJr SSKJrJ r J r J r SSK r SSK Jr SSK Jr SSKJr /rS\S\S \\\S 4\\S 444S jrS \ R.S-S\S\S \\\44S jrS\\S 4S\\S 4S \\\S 4\\\444Sjr\ "S\\\5r\ "S\ R:\5r\ S\S\ R>S\ S \\4Sj5r!\ S\S\ R>S\ S \\4Sj5r!Sr!S=S\S\S\ S S4Sjjr"S\ R:S\ RFS S4Sjr$S\ R:4S jr%\ "S!5r&\ "S"\\\\'\\\5r(\ S#\\ R:/\&4S$\ R:S \&4S%j5r)\ S#\\ R:/\4S$\(S \(4S&j5r)S'r)S\\S 4S\\\4S-S\ R>S\ S \\\S 4\\\\4S 444 S(jr*S>S)\RVS*\\ R:S+\ S,4S-jjr,S=S.\RZS)\RVS/\.S0\.S1\\S2\ S S4S3jjr/S)\RVS4\\ R:S/\.S0\.S S4 S5jr0S6\\\4S7\S8\S S4S9jr1S\ R:S \ 4S:jr2S;\\RZS \\RZ4S<jr3g)?N) OrderedDict)Callable Container)AnyOptionaloverloadTypeVar)nn)PackedSequenceargskwargsreturn.c/n[U5nUR5H'upEURU5 URU5 M) [U5[U54$)aJ Turn argument list into separate key list and value list (unpack_kwargs does the opposite). Inspiration: https://github.com/facebookresearch/fairscale/blob/eeb6684/fairscale/internal/containers.py#L70 Usage:: kwarg_keys, flat_args = pack_kwargs(1, 2, a=3, b=4) assert kwarg_keys == ("a", "b") assert flat_args == (1, 2, 3, 4) args, kwargs = unpack_kwargs(kwarg_keys, flat_args) assert args == (1, 2) assert kwargs == {"a": 3, "b": 4} Returns: Tuple[Tuple[Any, ...], Tuple[str, ...]]: The first tuple element gives gives both positional args and kwarg values, where the positional args proceed kwarg values and kwarg values are ordered consistently with the kwarg keys. The second tuple element gives the kwarg keys. The second tuple element's length is at most the first tuple element's length. )listitemsappendtuple)r r kwarg_keys flat_argskvs T/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/distributed/utils.py _pack_kwargsrsW(J:I !  U:. ..dtypec^TcX4$S[RS[R4U4Sjjn[X15[X254$)z Cast floating point tensors in ``args`` and ``kwargs`` to ``input_dtype``. This respects the existing ``requires_grad`` on the tensors. xrc>[R"U5(aURT:XaU$URT5$N)torchis_floating_pointrto)rrs rcast_fn%_cast_forward_inputs..cast_fn;s2&&q))QWW-=HttE{r)r Tensor_apply_to_tensors)rr r r#s` r_cast_forward_inputsr'.sE }|5<<ELL g ,.?.P QQrrrc [U5[U5:a#[S[U5S[U535e[U5S:XaU04$US[U5*n[[X[U5*S55nX#4$)zSee _pack_kwargs.ztoo many keys z vs. rN)lenAssertionErrordictzip)rrr r s r_unpack_kwargsr-Ds :Y'~c*o->eC NCSTUU :!"} 'J' (D #jS_,<,>"?@ AF <rSTinputs target_device!use_side_stream_for_tensor_copiescgrr0r1r2s r _recursive_tor6Usrcgrr4r5s rr6r6[src@^^^UUU4SjmT"U5nSmU$!Smf=f)z-Recursively moves input to the target_device.c>[U[R[45(Gau[U[5(aURR O UR nUT :XaU4$T (dUR T 54$URS:XaUR T 54$SSKJ n U"T 5nU UR T 5nSSS5 [RRT R5 [RR5nURU5 [U[5(aWRRU5 O;[W[R5(d [!S5eURU5 SSS5 U4$SSKJn U"U5(a/['[)T U56Vs/sHn[U5"U6PM sn$[U[*5(a*[-U5S:a[/['[)T U565$[U[.5(a;[-U5S:a,['[)T U56Vs/sHn[/U5PM sn$[U[05(aO[-U5S:a@['[)T UR3556Vs/sHn[U5"U5PM sn$U/$!,(df  GN=f!,(df  W4$=fs snfs snfs snf)Ncpur) _get_streamzoutput must be a torch.Tensor_is_namedtuple) isinstancer r%r datadevicer"typetorch.nn.parallel._functionsr; accelerator device_indexindexcurrent_stream wait_stream record_streamr* torch.nn.parallel.scatter_gatherr=r,maprr)rr+r) objr@r;streamoutputrFr=r ir1to_mapr2s rrO_recursive_to..to_mapds\ cELL.9 : :(23(G(GSXX__SZZF&v 4}-//;;%'FF=133D%]3 VVM2F&&33M4G4GH%*%6%6%E%E%GN"..v6"#~66 11.A)&%,,??"01P"QQ,,^<Iy C #  14cF1GH1GDIt$1GH H c5 ! !c#hlS-./ / c4 SX\%(#fc*:%;<%;DG%;< < c4 SX\*-s6399;/G*HI*HQDIaL*HI Iu ?VIHy I =Js+3K6BK'K:K?4L K$' K7Nr4)r0r1r2resrOs `` @rr6r6as*0fVn Jscondsraise_assertion_errorcxU(d3[U5 [R"5 U(a [U5egg)zwAlternate to ``assert`` when in the backward context to print the error message ``s`` since otherwise, it is swallowed.N)print traceback print_stackr*)rRrSrTs r _p_assertrYs0  a  # # ! rtensorsizec[R"5 [RRR 5(dUR 5R 5UR5:HnU(dZUR 5R 5n[US:HS5 UR 5RUR55 SSS5 g!,(df  g=f)z Allocate storage for ``tensor`` with the given size. Returns: bool: ``True`` if this method allocated storage and ``False`` if the storage was already allocated. rzCTensor storage should have been resized to be 0 but got PLACEHOLDERN) r no_grad distributed_functional_collectivesis_torchdynamo_compiling_typed_storage_sizenumelrY_resize_)rZr[already_allocatedtensor_storage_sizes r_alloc_storagergs   88QQSS & 5 5 7 = = ?4::< O $&,&;&;&=&C&C&E#'1,Y%%'00> s B>C C+c [R"5 [RRR 5(dUR 5R 5S:HnU(dy[UR5S:HSUR5SUR 5R 5SUR35 UR 5RS5 SSS5 g!,(df  g=f)z Frees the underlying storage of ``tensor``. Returns: bool: ``True`` if the method freed the storage and ``False`` if the storage was already freed. rzVFreeing a tensor's storage is unsafe when it is not the sole occupant storage offset: z storage size: z tensor shape: N) r r]r^r_r`rarbrYstorage_offsetshaperd)rZ already_freeds r _free_storagerls   88QQSS"11399;q@M ))+q0''-'<'<'>&?@%%+%:%:%<%B%B%D$EF%%+\\N4%%'003 s CC.. C<QRfn containercgrr4rorps rr&r&s rcgrr4rrs rr&r&sMPrc$^^UU4SjmT"U5$)zFRecursively apply to all tensor in different kinds of container types.c >SSKJn [U[R5(aT "U5$[ US5(av[ R"U5n[ R"U5Vs0sH(o3RT "[X#R55_M* nn[ R"U40UD6$[U[5(a5UR5nUR5HupgT "U5XV'M U$[U[5(aT "UR5 U$[U[ 5(a,UR5VVs0sHupgUT "U5_M snn$U"U5(aU 4SjU5n[#U5"U6$[U[$[&[(45(a[#U5"U 4SjU55$U$s snfs snnf)Nrr<__dataclass_fields__c34># UH nT"U5v M g7frr4.0elapplys r 3_apply_to_tensors..apply..s)q599qc34># UH nT"U5v M g7frr4rxs rr|r}s1q599qr~)rIr=r>r r%hasattr dataclassesreplacefieldsnamegetattrr __class__rr r?r+rArrset) rr=dcfchangesodkeyvaluerQr{ros rr{ _apply_to_tensors..applysyC a & &a5L Q. / /$$Q'B * * !&&MH 4 89 B *#Cu% B B A  )q)C7C= D%- . .71q11 1H)Cs ./GGr4)rorpr{s` @rr&r&s:  rcU(a [XU5O/nU(a [XU5O/n[U5[U5:aAUR[[U5[U5- 5Vs/sHnSPM sn5 OX[U5[U5:a@UR[[U5[U5- 5Vs/sHn0PM sn5 [ U5[ U54$s snfs snf)Nr4)r6r)extendranger)r0r r1r2 moved_inputs moved_kwargs_s r _to_kwargsrs  f-NO   f-NO   <3|,,ss* =A !! *? r state_dict old_prefix new_prefixcX:Xa [S5e[UR55H3nURU5(dMX#[ U5S-nXX'X M5 g)a Replace all keys that match a given old_prefix with a new_prefix (in-place). Usage:: state_dict = {"layer.xyz": torch.tensor(1)} replace_by_prefix_(state_dict, "layer.", "module.layer.") assert state_dict == {"module.layer.xyz": torch.tensor(1)} z*old_prefix and new_prefix must be distinctN) ValueErrorrkeys startswithr))rrrrnew_keys r_replace_by_prefixrKsaEFFJOO%&~~j)) 3z?#455(o  O 'rcDUR5R5S:$)Nr)untyped_storagedata_ptr)rZs r_data_ptr_allocatedrcs  ! ! # , , . 22rmodulesc/nUVs0sHo"[UR55_M nnUHQnSnUR5Hup&XBL=(a XF;nU(dMSn O U(dM@URU5 MS U$s snf)z Returns the modules in ``modules`` that are root modules (i.e. parent-less) with respect to the set ``modules``. In other words, these are the modules in ``modules`` that are the not child of any other module in ``modules``. TF)rrrr)r root_modulesrmodule_to_modulescandidate_moduleis_root_module_modulesis_child_modules r_get_root_modulesrgs%'L4;:4;&FNN$%%G:$ 1 7 7 9 F .O3C3O !& !: >    0 1$ :s#B)Tr)4rrW collectionsrcollections.abcrrtypingrrrr r torch.distributedr^rr torch.nn.utils.rnnr __all__rstrrrr'r+r-rr.r%r/r@boolr6rYSizergrlrmrrnr&r ProcessGrouprModuleintrrrrrr4rrrs-#/33 - //s/uU38_eCQTHo5U/V/:R ;; R RR 38_ R, S#X ,1#s(O  5c?DcN *+  CtU# C~.  #llOS !W    #llOS 1X  :z$C$C$$$?5<<?uzz?d?(4%,,4, CL CtUCncJ  %,,"# 05      P(ELL>3#67PAP!P P F4 #s(O4 cNT !4<<4(, 4  5c?E$sCx.#"56 67 44'+P$$P %,, P ] #P# W IIW$$WW W #,C. W  W W:  $$   %        S#X 03 33tBII4 ?r