"Ei#SSKrSSKJr SSKJrJrJr SSKJrJ r SSK r SSK J r SSK Jr SSKJr SSKJr SS KJrJr S S /r\ "S S S9r\"S5"SS \\55rSr\"S5"SS \55rg)N) namedtuple)CallableIteratorSized)AnyTypeVar)default_collate)functional_datapipe)dataframe_wrapper) IterDataPipe)_check_unpickable_fnvalidate_input_colCollatorIterDataPipeMapperIterDataPipe_T_coT) covariantmapc^\rSrSr%Sr\\S'\\S'S S\S\SS4U4SjjjrSr S\ \ 4S jr S\ 4S jrS rU=r$) rao Applies a function over each item from the source DataPipe (functional name: ``map``). The function can be any regular Python function or partial object. Lambda function is not recommended as it is not supported by pickle. Args: datapipe: Source Iterable DataPipe fn: Function being applied over each item input_col: Index or indices of data which ``fn`` is applied, such as: - ``None`` as default to apply ``fn`` to the data directly. - Integer(s) is used for list/tuple. - Key(s) is used for dict. output_col: Index of data where result of ``fn`` is placed. ``output_col`` can be specified only when ``input_col`` is not ``None`` - ``None`` as default to replace the index that ``input_col`` specified; For ``input_col`` with multiple indices, the left-most one is used, and other indices will be removed. - Integer is used for list/tuple. ``-1`` represents to append result at the end. - Key is used for dict. New key is acceptable. Example: >>> # xdoctest: +SKIP >>> from torchdata.datapipes.iter import IterableWrapper, Mapper >>> def add_one(x): ... return x + 1 >>> dp = IterableWrapper(range(10)) >>> # Invocation via functional form is preferred ... map_dp_1 = dp.map(add_one) >>> list(map_dp_1) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> # We discourage the usage of `lambda` functions as they are not serializable with `pickle` >>> # Use `functools.partial` or explicitly define the function instead >>> map_dp_2 = Mapper(dp, lambda x: x + 1) >>> list(map_dp_2) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] datapipefnNreturncR>[RRS5 [TU]5 Xl[ U5 X lX0lUcUb [S5e[U[[45(a[U5S:a [S5eUSnX@l[X#5 g)Nzpython.data_pipes.mapz3`output_col` must be None when `input_col` is None.z3`output_col` must be a single-element list or tupler)torch_C_log_api_usage_oncesuper__init__rr r input_col ValueError isinstancelisttuplelen output_colr)selfrrr r& __class__s e/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/utils/data/datapipes/iter/callable.pyrMapperIterDataPipe.__init__Hs $$%<=  R "  !7RS S j4- 0 0:" !VWW#AJ$2)c<^URcURcURT5$URcURT5nOr[UR[[ 45(a/[ U4SjUR55nUR"U6nOURTUR5n[T[ 5(aSn[ T5mOSnURci[UR[[ 45(a4UTURS'[ URSSSS9HnTU M OAUTUR'O1URS:XaTRU5 OUTUR'U(a [ T5$T$)Nc3.># UH nTUv M g7fN).0coldatas r) /MapperIterDataPipe._apply_fn..gs=nscnsTFrr)reverse)r r&rr"r#r$sortedappend)r'r2resargst_flagidxs ` r) _apply_fnMapperIterDataPipe._apply_fn`sI >> !doo&=774= >> !''$-C u 6 6=dnn==D''4.C''$t~~./C dE " "F:DF ?? "$..4-88*-T^^A&'!$.."4dCCS D(+T^^$"$ C (+T__%%uT{.$.r+c#X# URHnURU5v M g7fr.)rr=)r'r2s r)__iter__MapperIterDataPipe.__iter__s"MMD..& &"s(*c[UR[5(a[UR5$[ [ U5R S35e)Nz# instance doesn't have valid length)r"rrr% TypeErrortype__name__)r's r)__len__MapperIterDataPipe.__len__s@ dmmU + +t}}% %4:..//RSTTr+)rrr r&)NN)rE __module__ __qualname____firstlineno____doc__r __annotations__rrr=rrr@intrF__static_attributes__ __classcell__r(s@r)rrsn&PL  ** *  **0!/F'(5/'UUUr+c([UR5S:a [S5eUSn[R"U5n/n/nUHnXc;dM [S5e UHnX`;a"[ X5(d [S5eXnO!SSKJn URR5nUR[U55 U"X&5n URU 5 M [SU5n U "U6n U $![an [S5U eSn A ff=f)Nrz%Only supports one DataFrame per batchrzConversion keys mismatchz5Collate (DF)DataPipe requires callable as dict valuesz?unable to import default collation function from the TorchArrow CollateResult)r%items RuntimeError df_wrapper get_columnscallabletorcharrow.pytorchpytorchrecDefault Exceptionr8strr) conversionitemdf columns_name tuple_names tuple_valuesname collation_fntapevaluetpl_clsr$s r)_collate_helperrjs 4::BCC aB))"-LKL  #9: :  J,--"K&+L 0"ww0 3t9%RX&E")0+6G \ "E L "U s C66 D D  Dcollatec z^\rSrSrSr\S4S\S\S\4\ \ \-\\-4-S-S\S-SS4U4S jjjr S r U=r $) ra Collates samples from DataPipe to Tensor(s) by a custom collate function (functional name: ``collate``). By default, it uses :func:`torch.utils.data.default_collate`. .. note:: While writing a custom collate function, you can import :func:`torch.utils.data.default_collate` for the default behavior and `functools.partial` to specify any additional arguments. Args: datapipe: Iterable DataPipe being collated collate_fn: Customized collate function to collect and combine data or a batch of data. Default function collates to Tensor(s) based on data type. Example: >>> # xdoctest: +SKIP >>> # Convert integer data to float Tensor >>> class MyIterDataPipe(torch.utils.data.IterDataPipe): ... def __init__(self, start, end): ... super(MyIterDataPipe).__init__() ... assert end > start, "this example only works with end >= start" ... self.start = start ... self.end = end ... ... def __iter__(self): ... return iter(range(self.start, self.end)) ... ... def __len__(self): ... return self.end - self.start >>> ds = MyIterDataPipe(start=3, end=7) >>> print(list(ds)) [3, 4, 5, 6] >>> def collate_fn(batch): ... return torch.tensor(batch, dtype=torch.float) >>> collated_ds = CollateIterDataPipe(ds, collate_fn=collate_fn) >>> print(list(collated_ds)) [tensor(3.), tensor(4.), tensor(5.), tensor(6.)] Nrr^. collate_fnrc>Ub[TU]XS9 g[U5(a[TU]XS9 g[R"[ U5n[TU]XS9 g)N)r)rrrW functoolspartialrj)r'rr^rnr(s r)rCollatorIterDataPipe.__init__sZ  ! G X  5 ##  9'.. K   9r+r/)rErHrIrJrKr r rrdictr]rrNrOrPs@r)rrsp%X!&* ::S#X& sSy(S.( )* : tO : ::r+)rp collectionsrcollections.abcrrrtypingrrrtorch.utils.data._utils.collater %torch.utils.data.datapipes._decoratorr $torch.utils.data.datapipes.dataframer rU#torch.utils.data.datapipes.datapiper 'torch.utils.data.datapipes.utils.commonr r__all__rrrjrr/r+r)r}s"55 ;EP<  4(UoUe,oUoUd(VY::-:: ::r+