>:j >SSKJr SSKrSSKrSSKrSSKrSSKJrJr SSK J r SSK J r SSK Jr SSKJr SSKJr SSKrSS KJr SS KJrJrJrJr SS KJrJrJrJr SS KJ r SS K!J"r" SSK#J$r$ SSK%J&r& SSK'J(r( SSK)J*r* SSK+J,r, SSK-J.r.J/r/ SSK0J1r1 SSK2J3r3J4r4 SSK5J6r6 SSK7J8r8J9r9J:r:J;r;JJ?r?J@r@JArA SSKBJCrCJDrDJErEJFrF SSKGJHrH \E"5(aSSKIJJrJJKrKJLrLJMrMJNrN \R"\P5rQSSK#JRrR "SS\\5rTg!\Sa SrRNf=f)) annotationsN)ABCabstractmethod) OrderedDict)Callable) nullcontext)partial)Any)nn) BatchSampler ConcatDataset DataLoader RandomSampler)EvalPredictionPreTrainedTokenizerBaseTrainerTrainerCallback)FeatureExtractionMixin)BaseImageProcessor) WandbCallback)ProcessorMixin)TRAINING_ARGS_NAME)EvalLoopOutput)BaseDataCollator) BaseEvaluatorSequentialEvaluator BaseModel)BaseModelCardCallbackBaseModelCardData)Router)DefaultBatchSamplerGroupByLabelBatchSamplerMultiDatasetDefaultBatchSamplerNoDuplicatesBatchSamplerProportionalBatchSamplerRoundRobinBatchSampler)BaseTrainingArguments BatchSamplersMultiDatasetBatchSamplers)disable_loggingfullnameis_datasets_availableis_training_available)deprecated_kwargs)Dataset DatasetDictIterableDatasetIterableDatasetDictValue)TrackioCallbackcd^\rSrSrSr\r\r\ r \ r \ r\"SS9S$S%U4Sjjj5rS&S'SjjrS(SjrS&S)U4S jjjrS*S jrS+S jr\S,S j5rS-S.S jjrS/SjrS&S0U4SjjjrS1SjrS2S3U4SjjjrS2S4U4SjjjrS5SjrS&S6SjjrS7S8Sjjr S9S:Sjjr!S;Sjr"SSjr%SS?Sjjr&S@Sjr'SSASjjr(SBSjr)S&SCSjjr*\+SSDS jj5r,SESFS!jjr-S&SGU4S"jjjr.S#r/U=r0$)H BaseTrainer7u BaseTrainer is a simple but feature-complete training and eval loop for PyTorch based on the 🤗 Transformers :class:`~transformers.Trainer`. This trainer integrates support for various :class:`transformers.TrainerCallback` subclasses, such as: - :class:`~transformers.integrations.WandbCallback` to automatically log training metrics to W&B if `wandb` is installed - :class:`~transformers.integrations.TensorBoardCallback` to log training metrics to TensorBoard if `tensorboard` is accessible. - :class:`~transformers.integrations.CodeCarbonCallback` to track the carbon emissions of your model during training if `codecarbon` is installed. - Note: These carbon emissions will be included in your automatically generated model card. See the Transformers `Callbacks `_ documentation for more information on the integrated callbacks and how to write your own callbacks. Args: model (:class:`~sentence_transformers.base.model.BaseModel`, *optional*): The model to train, evaluate or use for predictions. If not provided, a `model_init` must be passed. args (:class:`~sentence_transformers.base.training_args.BaseTrainingArguments`, *optional*): The arguments to tweak for training. Will default to a basic instance of :class:`~sentence_transformers.base.training_args.BaseTrainingArguments` with the `output_dir` set to a directory named *tmp_trainer* in the current directory if not provided. train_dataset (Union[:class:`datasets.Dataset`, :class:`datasets.DatasetDict`, :class:`datasets.IterableDataset`, Dict[str, :class:`datasets.Dataset`]], *optional*): The dataset to use for training. Must have a format accepted by your loss function. eval_dataset (Union[:class:`datasets.Dataset`, :class:`datasets.DatasetDict`, :class:`datasets.IterableDataset`, Dict[str, :class:`datasets.Dataset`]], *optional*): The dataset to use for evaluation. Must have a format accepted by your loss function. loss (Optional[Union[:class:`torch.nn.Module`, Dict[str, :class:`torch.nn.Module`], Callable[[:class:`~sentence_transformers.base.model.BaseModel`], :class:`torch.nn.Module`], Dict[str, Callable[[:class:`~sentence_transformers.base.model.BaseModel`]]]], *optional*): The loss function to use for training. Can either be a loss class instance, a dictionary mapping dataset names to loss class instances, a function that returns a loss class instance given a model, or a dictionary mapping dataset names to functions that return a loss class instance given a model. In practice, the latter two are primarily used for hyper-parameter optimization. evaluator (Union[:class:`~sentence_transformers.base.evaluation.BaseEvaluator`, List[:class:`~sentence_transformers.base.evaluation.BaseEvaluator`]], *optional*): The evaluator instance for useful evaluation metrics during training. You can use an ``evaluator`` with or without an ``eval_dataset``, and vice versa. Generally, the metrics that an ``evaluator`` returns are more useful than the loss value returned from the ``eval_dataset``. A list of evaluators will be wrapped in a :class:`~sentence_transformers.base.evaluation.SequentialEvaluator` to run them sequentially. callbacks (List of [:class:`transformers.TrainerCallback`], *optional*): A list of callbacks to customize the training loop. Will add those to the list of default callbacks detailed in [here](callback). If you want to remove one of the default callbacks used, use the [`Trainer.remove_callback`] method. optimizers (`Tuple[:class:`torch.optim.Optimizer`, :class:`torch.optim.lr_scheduler.LambdaLR`]`, *optional*, defaults to `(None, None)`): A tuple containing the optimizer and the scheduler to use. Will default to an instance of :class:`torch.optim.AdamW` on your model and a scheduler given by :func:`transformers.get_linear_schedule_with_warmup` controlled by `args`. Important attributes: - **model** -- Always points to the core model. If using a transformers model, it will be a [`PreTrainedModel`] subclass. - **model_wrapped** -- Always points to the most external model in case one or more other modules wrap the original model. This is the model that should be used for the forward pass. For example, under `DeepSpeed`, the inner model is wrapped in `DeepSpeed` and then again in `torch.nn.DistributedDataParallel`. If the inner model hasn't been wrapped, then `self.model_wrapped` is the same as `self.model`. - **is_model_parallel** -- Whether or not a model has been switched to a model parallel mode (different from data parallelism, this means some of the model layers are split on different GPUs). - **place_model_on_device** -- Whether or not to automatically place the model on the device - it will be set to `False` if model parallel or deepspeed is used, or if the default `TrainingArguments.place_model_on_device` is overridden to return `False` . - **is_in_train** -- Whether or not a model is currently running `train` (e.g. when `evaluate` is called while in `train`) processing_class) tokenizerNNc >[5(d#[SURRS35eUcBSn[R SUR RSUS35 UR US9nO<[X R 5(d"[S[UR 5S 35eUc=U bXl UR5nOY[S URRS 35eU b-[RS URRS 35 Xl U b-[RS URRS35 UR SSS0S9R5nUR(a@UR R"(d%UR R%UR5 UcL['US5(a;[UR([*[,[.[045(a UR(nUcUR3XUS9n[5SS/X4/5Hunn[U[65(dMUR8bM,[;[=U55n[>S[@S[BS[DS0nURG5VVs0sH+unnU[IURK[MU5S55_M- nnn[SUSUSUSUS 3 5e [U[N5(a [U[P5(d [QU5n[U[N5(a [U[P5(d [QU5nUcCUc@URRS!:wa0[S"URRS#URRS$35e[TTU]UR(aSOUUUUUcUcUOS%UU U U U U US&9 URXS%:XaSUl,00S'.Ul-SUl.U U U [_S(UR`Rb55(a [dRfRiS)S*5 [jbK[_S+UR`Rb55(a [dRfRiS,S*5 UcURmURn5n[U[N5(aURG5VVs0sHunnUURqUU5_M snnUl9[5SS/X4/5HunnUcM [U[N5(d[S-US.35e[uURw55[uURw55- =n(dMk[S/US0[yU5S1[{U5S2:XaS3OS4S5US63 5e OURqXQ5Ul9Ub [U[|5(d [U5nX`l@URbURUSS79UlAURXbURUSS79Ul,URU5 gs snnfs snnf)8Nz To train a z model, you need to install the `accelerate` and `datasets` modules. You can do so with the `train` extra: pip install -U "sentence-transformers[train]" tmp_trainerzNo `args` passed, using `z (output_dir=z)`.) output_dirzPlease pass an instance of `z` as the `args` argument.`z4` requires either a `model` or `model_init` argumentz` requires either a `model` or `model_init` argument, but not both. `model_init` will overwrite your model when calling the `train` method.z7`compute_metrics` is currently not compatible with the z. Please use the `evaluator` argument instead for detailed evaluation metrics, or the `eval_dataset` argument for the evaluation loss.unuseduse_configured_stateT)r>accelerator_config processor)modelargsr9trainevalstringint64float32boolnullzThe provided `z6_dataset` must have Features. Specify them with e.g.: z _dataset = z_dataset.cast(Features(z)) or by providing the Features to the IterableDataset initialization method. See the Datasets documentation for more information on dataset Features: https://huggingface.co/docs/datasets/en/about_dataset_featuresnoz%You have set `args.eval_strategy` to zu, but you didn't provide an `eval_dataset` or an `evaluator`. Either provide an `eval_dataset` or an `evaluator` to `z7`, or set `args.eval_strategy='no'` to skip evaluation.dummy) rDrE data_collator train_dataset eval_datasetr9 model_initcompute_metrics callbacks optimizersoptimizer_cls_and_kwargspreprocess_logits_for_metrics)rFrGc3B# UHn[U[5v M g7fN) isinstancer.0callbacks _/home/mande/repo/quber/.venv/lib/python3.13/site-packages/sentence_transformers/base/trainer.py 'BaseTrainer.__init__..scCbxz(M22Cb WANDB_PROJECTzsentence-transformersc3B# UHn[U[5v M g7frY)rZr5r[s r^r_r`s/ BahJx 1 1BaraTRACKIO_PROJECTz,If the provided `loss` is a dict, then the `z"_dataset` must be a `DatasetDict`.z:If the provided `loss` is a dict, then all keys from the `z;_dataset` dictionary must occur in `loss` also. Currently, z occursz in `z_dataset` but not in `loss`. dataset_name)Dr. RuntimeError model_class__name__loggerinfotraining_args_classrZ ValueErrorr,rRcall_model_init __class__warningto_dict hub_model_idmodel_card_datamodel_id set_model_idhasattrrCrrrrget_data_collatorzipr2 column_namesnextiterstrintfloatrKitemsr4gettypedictr1 eval_strategysuper__init__rQaccum_loss_componentscan_return_lossanycallback_handlerrTosenviron setdefaultr5get_default_lossrD prepare_losslosssetkeyssortedlenrr evaluatorrPpreprocess_datasetadd_model_card_callback)selfrDrErPrQrrrOr9rRrSrTrUrVrWr>default_args_dictridatasetsamplenaive_type_mappingkeyvalueexample_featuresloss_fnmissingrrs r^rBaseTrainer.__init__sh4%&&d..7789@@  <&J KK3D4L4L4U4U3VVbcmbnnqr s++z+BDD":":;;.x8P8P/Q.RRkl  =%",,,."Qt~~'>'>&??s#tuu%//01^^)O  & NNI$..JaJaIbc'' !445KT4R5 ')    U%:%:%C%C  ! ! . .t/@/@ A  ${++"9;MOegu!v %    22\l2mM%('6):]" $    c4CXCXCbCbc c c JJ ! !/3J K  &3/ BFBWBWBaBa/ , ,  JJ ! !"35L M <((4D dD ! !fjfpfpfrsfrMb\[bt'8'8%'HHfrsDI),gv->@])^% g?!'400$F|nTvw"',,.1C 4DDD7D$TUaTbc&&,Wo%6fCLTUDUS[]<^^cdpcqrNO*_))$6DI  I})M)M+I6I"    )!%!8!8U\!8!]D     ( $ 7 7 SY 7 ZD  $$%67y$Fts *2Y- Y c[UR5Vs/sHoDRPM sn;aUR(d [ S5eUR UR URURS9$s snf)a  Load the data collator for the trainer. Args: model (:class:`~sentence_transformers.base.model.BaseModel`): The model to train, evaluate or use for predictions. args (:class:`~sentence_transformers.base.training_args.BaseTrainingArguments`): The arguments to tweak for training. processing_class (Union[:class:`transformers.PreTrainedTokenizerBase`, :class:`transformers.BaseImageProcessor`, :class:`transformers.FeatureExtractionMixin`, :class:`transformers.ProcessorMixin`], *optional*): The processing class to use for tokenization or image processing. Returns: :class:`BaseDataCollator`: The data collator to use for the trainer .. note:: This method can be overridden by subclassing the trainer to use a custom data collator. alYou are using a Router module in your model, but you did not provide a `router_mapping` in the training arguments. This means that the Router module will not be able to route the inputs to the correct submodules. Please provide a `router_mapping` that maps column names to routes, e.g. {'column_one': 'query', 'column_two': 'document', 'column_three': 'document'}.) preprocess_fnrouter_mappingprompts)r!childrenrrrrpdata_collator_class preprocessr)rrDrEr9modules r^rzBaseTrainer.get_data_collator8s|6 U^^5EF5E6&&5EF FtObObf ''**..LL(  GsA>cURU5nURU5 URURURUR UR US9 g)a Add a callback responsible for automatically tracking data required for the automatic model card generation This method is called in the ``__init__`` method of the trainer subclass. Args: default_args_dict (Dict[str, Any]): A dictionary of the default training arguments, so we can determine which arguments have been changed for the model card. .. note:: This method can be overridden by subclassing the trainer to remove/customize this callback in custom uses cases )rDtrainerN)model_card_callback_class add_callback on_init_endrEstatecontrolrD)rrmodel_card_callbacks r^r#BaseTrainer.add_model_card_callbackasR#<<=NO -.'' 4::t||SWS]S]gk'lc>[TU]US9n[US5(dU$[UR[ 5(aURR 5Hvup4[U[RR5(dU"U5URU'ME[US5(dMXURXB5URU'Mx U$[UR[RR5(dUR U5UlU$[URS5(a!URURU5UlU$)N)trialrrD) rrqryrZrrrtorchr Moduleoverride_model_in_loss)rrrDrrrrs r^rqBaseTrainer.call_model_initts'e'4tV$$L dii & & $  1 !'588??;;%,U^DIIcNWg..%)%@%@%PDIIcN !2 DIIuxx77 %(DI  TYY ( (33DIIuEDI rc  SSKJn UR5HiupEUS:Xa[XS5(aX!lM#[U[ R R5(dMN[XURXR55 Mk U$)NrrrD) sentence_transformers.base.modelrnamed_childrenrZrDrr rsetattrr)rrrDrnamechilds r^r"BaseTrainer.override_model_in_losssa>..0KDw:e#?#?" E588??33D$?$?$MN 1  rc[U[RR5(aUR UR 5nO!U"U5R UR 5n[ USS5(aq[US[5(a0USRR5Vs/sHo3SPM nnOUS/nUHn[US5(dMSUl M U$s snf)Nrequires_media_countsFrtrack_media_countsT) rZrr rtodevicegetattrr! sub_modulesvaluesryr)rrrDroute input_modulesrs r^rBaseTrainer.prepare_losss dEHHOO , ,775<<(D;>>%,,/D 40% 8 8%(F++7AEA[A[\iAjknAorwAw**=9#>' $44]C CAFaWZchcocoApD & &} 5g > ""=1':a?:rc >SnSU;aSnOSU;aSnU(GaRUR5n[US5(aURURU5nO.SSKJn U"URUUR RS9nS U;aURS 5R5R5nURUS ==S-ss'UR5HupxUS :XaM US:XaUS U3OUn [UR5U- R5S 5X'[R"S URUR S 9URUU'M Ub["T U]IX5$["T U]IU5$)NrrF eval_lossrG_nested_gatherr) nested_gather) parallel_moder_rr)copyryrrtransformers.trainer_pt_utilsrrErrritemrroundrrrrrlog) rlogs start_timer accum_lossesrrrrlog_keyrrs r^rBaseTrainer.logsy T>#M D "M 99;Dt-..#2243M3Mm3\] G,..}=TYYMdMd  ,&$((1557<<>**=9'BaGB"."4"4"6JCg~ :G6:Qq6WZG$)599;+>*D*D*F$JDMEJ\\5;;u||FD..}=cB #7  !7;t0 07;t$ $rc Sn/n[5nUHnSnUH-nURSU-5(dMUS[U5*n O UbXd;aMBURU5 UR UR 5VV s0sH,upUR U5(dMU[U5SU _M. sn n5 M URSS5n X:4$s sn nf)asTurn the inputs from the dataloader into the separate model inputs & the labels. Example:: >>> list(inputs.keys()) ['return_loss', 'label', 'sentence_0_input_ids', 'sentence_0_token_type_ids', 'sentence_0_attention_mask', 'sentence_1_input_ids', 'sentence_1_token_type_ids', 'sentence_1_attention_mask'] >>> features, labels = self.collect_features(inputs) >>> len(features) 2 >>> list(features[0].keys()) ['input_ids', 'token_type_ids', 'attention_mask'] >>> list(features[1].keys()) ['input_ids', 'token_type_ids', 'attention_mask'] >>> torch.equal(labels, inputs["label"]) True ) input_idssentence_embedding pixel_valuesinput_features input_valuespixel_values_videosNrlabel)rendswithraddappendr startswithr) rrfeature_suffixesr seen_prefixescolumnprefixsuffixrrrs r^rBaseTrainer.collect_features#s(  FF*??3<00#Ns6{l3F+~!8   f % OOr:3[^[i[ijp[q6SV/6r sGT*ss C "C cd>UbURUSS9nO URn[TU] XU5$)NrGrh)rrQrevaluate)rrQ ignore_keysmetric_key_prefixrrs r^rBaseTrainer.evaluateNs>  #22[T U]UUUUUS9nURcU$UR(ag[ UR [ 5(aHURS5(a2USS[UR R55S:XaSnOU$UR5(a [5O[[R5 URR nUb5["R$R'US5n["R("USS9 URUR*XpR,R.UR,R0S9nSSS5 [ W[ 5(dS U0n[UR55H6n U RUS 35(aMUR3U 5XS U 3'M8 UR4R7U5 U$!,(df  N=f) N) dataloader descriptionprediction_loss_onlyrreval_rrGTexist_ok) output_pathepochrrr)revaluation_loopr is_in_trainrZrQrr rris_local_process_zerorr+loggingINFOrEr>rpathjoinmakedirsrDrr!rrmetricsupdate) rrrrrroutputr evaluator_metricsrrrs r^r"BaseTrainer.evaluation_loopZs(!#!5#/ )  >> !M    4+<+->-C-C-E(Fq(II$*! "88::[]PWP\P\@] ]))..K& ggll;?  K$7 $ ::;K;KSWS]S]SiSi!/!  ^+T22!,.? @ )..01C>>%6$7q"9::BSBWBWX[B\!$7q">?2 /0 %^ ]s BG55 Hc X[RSURRSURRS35 URR=n(aCUR SS5SnUR RR[U55 URURR5 g![a N3f=f![aBn[RSURRS[U535 SnAgSnAff=f) NzLoading best model from z (score: z).-rez#Could not load the best model from z . Error: )rmrnrbest_model_checkpoint best_metricrsplitrDrvset_best_model_stepr Exception_load_from_checkpointerrorr)r checkpointstepexcs r^_load_best_modelBaseTrainer._load_best_models .tzz/O/O.PPYZ^ZdZdZpZpYqqstu !ZZ===z=!((a04 **>>s4yI   & &tzz'G'G H       LL>tzz?_?_>``ijmnqjrist u  s+A C '%C CC D)'8D$$D)c[U[5(a)UR5Hup!URXS9 M g[ UR 5SS1-=n(a'[ SU(aUS-OSS[U5S35eg) Nrh return_lossriz/The following column names are invalid in your  rgz dataset: zH. Avoid using these column names, as they are reserved for internal use.)rZrrvalidate_column_namesrr|rpr)rrrioverlaps r^rA!BaseTrainer.validate_column_namess gt $ $)0% **7*N*9 '../=.2QQ Q7 QAXd,QTBTjlAmmvw{}DxEwFFZZ  RrcUUUUUS.n[R"URR5(aF[ URR[ 5(aURR"U40UD6$[ URR5(aURR"U40UD6$[U[5(a>URR[R:wa[RS5 gURR[R:Xa [U40UD6$URR[R:Xa[U4SS0UD6$URR[R :Xa [#U40UD6$URR[R:Xa[ [%XS940UD6$g)a Returns the appropriate batch sampler based on the ``batch_sampler`` argument in ``self.args``. This batch sampler class supports ``__len__`` and ``__iter__`` methods, and is used as the ``batch_sampler`` to create the :class:`torch.utils.data.DataLoader`. .. note:: Override this method to provide a custom batch sampler. Args: dataset (Dataset): The dataset to sample from. batch_size (int): Number of samples per batch. drop_last (bool): If True, drop the last incomplete batch if the dataset size is not divisible by the batch size. valid_label_columns (List[str]): List of column names to check for labels. The first column name from ``valid_label_columns`` found in the dataset will be used as the label column. generator (torch.Generator, optional): Optional random number generator for shuffling the indices. seed (int): Seed for the random number generator to ensure reproducibility. Defaults to 0. ) batch_size drop_lastvalid_label_columns generatorseedBWhen using an IterableDataset, you cannot specify a batch sampler.Nprecompute_hashesT)rH)inspectisclassrE batch_sampler issubclassr"callablerZr2r) BATCH_SAMPLERrmrs NO_DUPLICATESr%NO_DUPLICATES_HASHEDGROUP_BY_LABELr#r)rrrErFrGrHrIbatch_sampler_kwargss r^get_batch_samplerBaseTrainer.get_batch_samplers>%"#6"   ??49922 3 3 499CZCZ\o8p8p99**7K6JK K DII++ , ,99**7K6JK K g / /yy&&-*E*EEcd 99 " "m&A&A A+GL7KL L 99 " "m&H&H H+GdtdOcd d 99 " "m&B&B B+GL7KL L 99 " "m&A&A A&}W'RkVjk k BrcPUUUS.n[R"URR5(aF[ URR[ 5(aURR"U40UD6$[ URR5(aURR"U40UD6$URR[R:Xa [SSU0UD6$URR[R:Xa [SSU0UD6$g)a Returns the appropriate multi-dataset batch sampler based on the ``multi_dataset_batch_sampler`` argument in ``self.args``. This batch sampler class supports ``__len__`` and ``__iter__`` methods, and is used as the ``batch_sampler`` to create the :class:`torch.utils.data.DataLoader`. .. note:: Override this method to provide a custom multi-dataset batch sampler. Args: dataset (ConcatDataset): The concatenation of all datasets. batch_samplers (List[BatchSampler]): List of batch samplers for each dataset in the concatenated dataset. generator (torch.Generator, optional): Optional random number generator for shuffling the indices. seed (int, optional): Optional seed for the random number generator )batch_samplersrHrIrNr) rLrMrEmulti_dataset_batch_samplerrOr$rPr* ROUND_ROBINr' PROPORTIONALr&)rrrYrHrImulti_batch_sampler_kwargss r^get_multi_dataset_batch_sampler+BaseTrainer.get_multi_dataset_batch_samplers.-"& " ??499@@ A Aj II 1 13RG G 9988_D^_ _ DII99 : :9988_D^_ _ 99 0 04M4Y4Y Y)X'X=WX X 99 0 04M4Z4Z Z+ZGZ?YZ Z [rc >URn[R"5nURRb%UR URR5 UURR URRURRURRS.n[U[5(agURUURRS.5 URR[R :wa["R%S5 GO\[U[&5(a [)S5e[U[*5(aUR-5H#n[U[5(dM[)S5e UR-5Vs/sH4nUR/UUURRUR0US9PM6 nn[3UR-55nUR5UUUURRS9n XS'O][U[65(a6UR/UUURRUR0US9n XS'O[)S US US 35e[9U40UD6$s snf) a#Shared logic for building train/eval/test DataLoaders. Args: dataset: The dataset to build a DataLoader for. batch_size: The batch size to use. dataset_kind: A label for error messages, e.g. "train", "eval", or "test". Returns: A prepared DataLoader for the given dataset. ) collate_fn num_workers pin_memorypersistent_workersprefetch_factor)rErFrJzcSentence Transformers is not compatible with IterableDatasetDict. Please use a DatasetDict instead.zYSentence Transformers is not compatible with a DatasetDict containing an IterableDataset.)rErFrGrH)rrYrHrIrNz Unsupported `zC_dataset` type. Use a Dataset, DatasetDict, or IterableDataset for .)rOr GeneratorrErI manual_seeddataloader_num_workersdataloader_pin_memorydataloader_persistent_workersdataloader_prefetch_factorrZr2r+dataloader_drop_lastrNr)rQrmrsr3rpr1rrVrGr r^r0r) rrrE dataset_kindrOrHdataloader_params sub_datasetrYrNs r^_build_dataloaderBaseTrainer._build_dataloadersb ** OO% 99>> %  ! !$)).. 1(99;;))99"&))"I"I#yyCC   g / /  $ $",!%!?!?  yy&&-*E*EEcd !4 5 5u  - -&~~/ k?;;$s 0$+>>#3 $4K&&)"ii<<(5(I(I' '$4  $GNN$45G @@-#YY^^ AM 2?o .  ) ) 22%))88$1$E$E# 3M2?o . ~-pq}p~~A '7%677E s0;Jc6URc#[SURRS35eSURlURR URURURRSS95Ul UR$)a Returns the training [`~torch.utils.data.DataLoader`]. Will use no sampler if `train_dataset` does not implement `__len__`, a random sampler (adapted to distributed training if necessary) otherwise. Subclass and override this method if you want to inject some custom behavior. z4Training requires specifying a train_dataset to the rfFrFrn) rPrprrrl accelerator even_batchespreparerqrEtrain_batch_size_train_dataloader)rs r^get_train_dataloader BaseTrainer.get_train_dataloaderms    %STXTbTbTkTkSllmno o ).%!%!1!1!9!9  " "4#5#5tyy7Q7Q`g " h" %%%rcVUcHURc;URb [/5$[SURR S35eUbUO URnSUR lUR RURXRRSS95$)a Returns the evaluation [`~torch.utils.data.DataLoader`]. Subclass and override this method if you want to inject some custom behavior. Args: eval_dataset (`torch.utils.data.Dataset`, *optional*): If provided, will override `self.eval_dataset`. If it is a [`~datasets.Dataset`], columns not accepted by the `model.forward()` method are automatically removed. It must implement `__len__`. z6Evaluation requires specifying an eval_dataset to the rfTrGrt) rQrrrprrrlrurvrwrqrEeval_batch_size)rrQs r^get_eval_dataloaderBaseTrainer.get_eval_dataloaders  D$5$5$=~~)!"~%UVZVdVdVmVmUnnopq q'3'?|TEVEV )-%''  " "<1J1JY_ " `  rcSURlURRURXRR SS95$)a} Returns the test [`~torch.utils.data.DataLoader`]. Subclass and override this method if you want to inject some custom behavior. Args: test_dataset (`torch.utils.data.Dataset`, *optional*): The test dataset to use. If it is a [`~datasets.Dataset`], columns not accepted by the `model.forward()` method are automatically removed. It must implement `__len__`. Ttestrt)rurvrwrqrEr})r test_datasets r^get_test_dataloaderBaseTrainer.get_test_dataloadersI)-%''  " "<1J1JY_ " `  rc2UbUOURRn[R"USS9 [R SU35 [ URS5(a.URRXRRS9 OURRU5 URbURRU5 [R"UR[RRU[55 g)NTrzSaving model checkpoint to save_safetensors)safe_serialization)rEr>rr)rmrnryrDsave_pretrainedrr9rsaver'r(r)rr> state_dicts r^_saveBaseTrainer._saves#-#9Ztyy?S?S  J. 1*>? 4990 1 1 JJ & &ziiF`F` & a JJ & &z 2  ,  ! ! 1 1* =  499bggll:7IJKrcURRnU"XRRS9nURRUR 55 g)N)trust_remote_code)rDrrrload_state_dictr)rcheckpoint_pathrk loaded_models r^r7!BaseTrainer._load_from_checkpoints@jj** "?jjFbFbc  ""<#:#:#<=rc[US5(dUcU$URXS9 URXRUR5(aUR U5nSUlU$)a Preprocess the dataset by optionally lazily adding a dataset name column, required for multi-dataset training with multiple losses or for dataset-specific router mappings. Args: dataset (DatasetDict | Dataset | None): The dataset to preprocess. If None, no preprocessing is done. dataset_name (str | None): The name of the dataset, used for multi-dataset training with multiple losses. Returns: DatasetDict | Dataset | None: The preprocessed dataset, perhaps with dataset names added as a lazy column. #_sentence_transformers_preprocessedrhT)ryrA#should_dataset_name_column_be_addedrEradd_dataset_name_columnr)rrris r^rBaseTrainer.preprocess_datasetsh 7A B BgoN ""7"F  3 3GYY R R227;G7;3rc[U[[45=(a [U[5=(d UR=(a [UR[5=(dn UR =(a[ [UR [5=(a: [[ [UR R555[5$)z We should add a dataset name column to the dataset, if the dataset is a DatasetDict, *and* one of: a. The loss is a dictionary, or b. The prompts contain a mapping of dataset names, or c. The router_mapping contains a mapping of dataset names. ) rZr1r3rrrr}r~r)rrrErs r^r/BaseTrainer.should_dataset_name_column_be_addeds'K1D#EF tT "  ?DLL$!? ##Ot22D9OtD)<)<)C)C)E$FGN  rc[U[[45(a-UR5Hup#UR UUS9X'M U$UcU$[U[ 5(a3UR [UR4SU0URD65 U$[U[5(aFURnU(a[S5US'UR[URUS9SUS9nU$[S5e)N)rririrHrhT)batchedrz`Unsupported `dataset` type. Use a Dataset, DatasetDict, IterableDataset, or IterableDatasetDict.)rZr3r1rrr0 set_transformr add_dataset_name_transform_format_kwargsr2rr4maprp)rrri inner_datasetrs r^r#BaseTrainer.add_dataset_name_columns g 3[A B B/6}}+ (,(D(D)!-)E)%0? N  N gw ' '  ! !33!-,, 6% 1 1''H+0?(kk33!-! "Gr rc U(aU"U5nU(a$[UR55S(aUcU$[[UR55S5nU/U-US'U$)aA transform/map function that adds the dataset name to the batch. Args: batch (dict[str, list[Any]]): The batch of data, where each key is a column name and each value is a list of values. dataset_name (str | None, optional): The name of this dataset, only if there are multiple datasets that use a different loss. Defaults to None. transform (Callable[[dict[str, list[Any]]], dict[str, list[Any]]], optional): An optional transform function to apply on the batch before adding the dataset name. Defaults to None. Returns: dict[str, list[Any]]: The "just-in-time" transformed batch with the dataset name added. rri)rrr)batchri transformkwargsrEs r^r&BaseTrainer.add_dataset_name_transform.sd, e$ED03|7KLelln-a01 !- ;n rc  UR5(dgU(a%URRRU5 U(a%URRR U5 U(a%URRR U5 URR URRUS9 g)N) model_name) is_world_process_zerorDrv set_language set_licenseadd_tags_create_model_cardrEr>) rlanguagelicensetagsrfinetuned_fromtasks dataset_tagsr dataset_argsrs r^create_model_cardBaseTrainer.create_model_cardPs))++   JJ & & 3 3H =  JJ & & 2 27 ;  JJ & & / / 5 %%dii&:&:z%Rrc >^[UR[5(a*[R"[ UR55nO URn[ TU]X5upEURU5n1Sk[UR55-(dUR5VVs/sH!upxXv;dM UR(dMUPM# snnURRS.UR5VVs/sH!upxXv;dM UR(dMUPM# snnSS./US'URR!5GHup[UR551Sk-n U (aU R#5OSn UR5VVs0sH$upx[$R&"X5(dM"Xx_M& n nnU (aXX\HOnSU;dM USV^s/sH/m[)U4SjU R+555(dM-TPM1 snUS'MQ O[-SU S35eU R!5VVs0sHupxXv;dM Xx_M nnnU R!5VVs0sHupxXv;dM Xx_M nnnU(aBX\R/[1UR+55U URRS .5 U(dGMX\R/[1UR+55U SS .5 GM XE4$s snnfs snnfs snnfs snfs snnfs snnf) a  We have to override the optimizer_grouped_parameters because the Trainer superclass bases it on the `model` itself, but the BaseModel losses can have weights that should be updated as well, e.g. SoftmaxLoss (see #2872). This method requires `transformers` >= 4.43.0. >rDparamsoptimizer_dict)r weight_decayrrrc3,># UH nTULv M g7frYr)r\paramps r^r_;BaseTrainer.get_optimizer_cls_and_kwargs..s=sZrQVaunZrsz*No parameters found matching the pattern 'z^' in the model. Please check the pattern and ensure it matches some of the model's parameters.)rlrr)rZrrr Sequentialrrget_optimizer_cls_and_kwargsget_decay_parameter_namesrrnamed_parameters requires_gradrErlearning_rate_mappingrrresearchallrrpr r)rrErD loss_model optimizer_clsoptimizer_kwargsdecay_parametersnrparameter_pattern learning_rateoptimizer_param_keysoptimizer_param_keymatching_paramsgroupmatching_params_with_decaymatching_params_without_decayrrs ` r^r(BaseTrainer.get_optimizer_cls_and_kwargsis dii & &{499'=>JJ*/'*Nt*`'  99*E4s;K;P;P;R7SS'1&A&A&C&CdaH]bcbqbq&C%)II$:$: '1&A&A&C&CdaHafgfufu&C%(  2 - . 150J0J0P0P0R , #&'7'<'<'>#?Bg#g @T"6":":"M>S>S>U,s>UdaYZYrTQT>U ),s) 5<<"&'A'H'H'J"K+(, (>(>-, 5<<"&'D'K'K'M"N+(+I1SX..u r +*m,ssT0 L?LL L#L#'L#"!L)L).,L/L/ L4 L4< L: L:)ryrrrQrrrRrP)NNNNNNNNNNNr;NN)rDBaseModel | NonerEzBaseTrainingArguments | NonerPCDataset | DatasetDict | IterableDataset | dict[str, Dataset] | NonerQrrznn.Module | dict[str, nn.Module] | Callable[[BaseModel], torch.nn.Module] | dict[str, Callable[[BaseModel], torch.nn.Module]] | Nonerz*BaseEvaluator | list[BaseEvaluator] | NonerOzBaseDataCollator | Noner9]PreTrainedTokenizerBase | BaseImageProcessor | FeatureExtractionMixin | ProcessorMixin | NonerRzCallable[[], BaseModel] | NonerSz'Callable[[EvalPrediction], dict] | NonerTzlist[TrainerCallback] | NonerUz?tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]rVz9tuple[type[torch.optim.Optimizer], dict[str, Any]] | NonerWz;Callable[[torch.Tensor, torch.Tensor], torch.Tensor] | NonereturnNonerY)rDrrEr(r9rrr)rzdict[str, Any]rr)rr)rtorch.nn.ModulerDrrr)rz8Callable[[BaseModel], torch.nn.Module] | torch.nn.ModulerDrrr)rDrrr)FN)rDrrdict[str, torch.Tensor | Any]rrKrz2torch.Tensor | tuple[torch.Tensor, dict[str, Any]])rzdict[str, torch.Tensor]rr)rdict[str, float]rz float | Nonerr)rrrz9tuple[list[dict[str, torch.Tensor]], torch.Tensor | None])NNrG)rQz#Dataset | dict[str, Dataset] | Nonerlist[str] | Nonerrrr) rrrrrz bool | Nonerrrrrr)rr)rr0ri str | Nonerr)NNr)rr0rErrFrKrGrrHtorch.Generator | NonerIrrzBatchSampler | None)Nr) rr rYzlist[BatchSampler]rHrrIz int | Nonerr )r'Dataset | DatasetDict | IterableDatasetrErrnrrr)rr)rQz.Dataset | DatasetDict | IterableDataset | Nonerr)rrrr)r>rrr)rrrr)rDatasetDict | Dataset | Nonerirrr)rrrEr(rz nn.Module | dict[str, nn.Module]rrK)rz=DatasetDict | IterableDatasetDict | Dataset | IterableDatasetrirrr)rdict[str, list[Any]]rirrz=Callable[[dict[str, list[Any]]], dict[str, list[Any]]] | Nonerr) NNNNNNNNN)rrrrrstr | list[str] | Nonerrrrrrrrrrrrrr)rEr(rDrrztuple[Any, Any])1rl __module__ __qualname____firstlineno____doc__rrkr model_card_data_classrrrrr(ror/rrzrrqrrrrrrrrrr"r<rArVr^rqrzr~rrr7rrr staticmethodrrr__static_attributes__ __classcell__)rrs@r^r7r77s@DK- 5*/!34#'-1]a\` @D15 59CG26Vb^bei/u8u8+u8[ u8 Z u8  u8>u8/u8u8$3%u8&A'u8(0)u8*T+u8,#\-u8.(c/u80 1u85u8~' ' $'  '  ' Rm&2F  ,   % 22.2 2 < 2h@*'%'%R) 3) B) Z=A(,!' N9 N& N N  N N -1(,!' ///* / & /  / //b" "15,0?l?l?l ?l . ?l * ?l?l ?lJ-1 *[*[+*[* *[  *[  *[XV88V8V8 V8  V8p&* 6 &L(> X\3JT %< - $ /    4$(.N.!. & .`$(SW# Q  F $"'+!%%)(,/3*./3SSS% S  S # S&S-S(S-S S4FJQ/)Q/2BQ/ Q/Q/rr7)U __future__rrLr%rrabcrr collectionsrcollections.abcr contextlibr functoolsr typingr rr torch.utils.datar r rr transformersrrrr%transformers.feature_extraction_utilsr#transformers.image_processing_utilsrtransformers.integrationsrtransformers.processing_utilsrtransformers.trainerrtransformers.trainer_utilsr(sentence_transformers.base.data_collatorr%sentence_transformers.base.evaluationrrrr%sentence_transformers.base.model_cardrr "sentence_transformers.base.modulesr!"sentence_transformers.base.samplerr"r#r$r%r&r'(sentence_transformers.base.training_argsr(r)r*sentence_transformers.utilr+r,r-r.%sentence_transformers.util.decoratorsr/datasetsr0r1r2r3r4 getLoggerrlrmr5 ImportErrorr7rrr^rs" ##$" SSZZHB3835ET6Z5utnnCZZ   8 $9 C/'3C/ Os>DDD