>:jSSKJr SSKJr SSKrSSKJr SSKJr SSK J r SSK J r JrJr SSKJr \ R$"\5r"S S \5rg) ) annotations)AnyN) Tokenizer)PreTrainedTokenizerBase)logging)Modality PairInput SingleInput)Modulec\rSrSr%SrSrS\S'S\S'\SSj5r\ SS j5r SSS jjr SS jr SS jr Srg ) InputModuleaq Subclass of :class:`sentence_transformers.base.modules.Module`, base class for all input modules in the Sentence Transformers library, i.e. modules that are used to process inputs and optionally also perform processing in the forward pass. This class provides a common interface for all input modules, including methods for loading and saving the module's configuration and weights, as well as input processing. It also provides a method for performing the forward pass of the module. Two abstract methods are inherited from :class:`~sentence_transformers.base.modules.Module` and must be implemented by subclasses: - :meth:`sentence_transformers.base.modules.Module.forward`: The forward pass of the module. - :meth:`sentence_transformers.base.modules.Module.save`: Save the module to disk. Additionally, subclasses should override: - :meth:`sentence_transformers.base.modules.InputModule.preprocess`: Preprocess the inputs and return a dictionary of preprocessed features. Optionally, you may also have to override: - :attr:`sentence_transformers.base.modules.InputModule.modalities`: The list of supported input modalities. Defaults to ``["text"]``. Override this to advertise support for non-text modalities (e.g. ``["text", "image"]``). - :meth:`sentence_transformers.base.modules.Module.load`: Load the module from disk. To assist with loading and saving the module, several utility methods are provided: - :meth:`sentence_transformers.base.modules.Module.load_config`: Load the module's configuration from a JSON file. - :meth:`sentence_transformers.base.modules.Module.load_file_path`: Load a file from the module's directory, regardless of whether the module is saved locally or on Hugging Face. - :meth:`sentence_transformers.base.modules.Module.load_dir_path`: Load a directory from the module's directory, regardless of whether the module is saved locally or on Hugging Face. - :meth:`sentence_transformers.base.modules.Module.load_torch_weights`: Load the PyTorch weights of the module, regardless of whether the module is saved locally or on Hugging Face. - :meth:`sentence_transformers.base.modules.Module.save_config`: Save the module's configuration to a JSON file. - :meth:`sentence_transformers.base.modules.Module.save_torch_weights`: Save the PyTorch weights of the module. - :meth:`sentence_transformers.base.modules.InputModule.save_tokenizer`: Save the tokenizer used by the module. - :meth:`sentence_transformers.base.modules.Module.get_config_dict`: Get the module's configuration as a dictionary. And several class variables are defined to assist with loading and saving the module: - :attr:`sentence_transformers.base.modules.Module.config_file_name`: The name of the configuration file used to save the module's configuration. - :attr:`sentence_transformers.base.modules.Module.config_keys`: A list of keys used to save the module's configuration. - :attr:`sentence_transformers.base.modules.InputModule.save_in_root`: Whether to save the module's configuration in the root directory of the model or in a subdirectory named after the module. - :attr:`sentence_transformers.base.modules.InputModule.tokenizer`: The tokenizer used by the module. Tbool save_in_rootz#PreTrainedTokenizerBase | Tokenizer tokenizercS/$)zAThe list of supported input modalities. Defaults to ``["text"]``.text)selfs l/home/mande/repo/quber/.venv/lib/python3.13/site-packages/sentence_transformers/base/modules/input_module.py modalitiesInputModule.modalitiesEs xc6UVs/sHo!U-PM sn$s snf)z+Prepend a prompt string to each text input.r)inputspromptrs r_prepend_promptInputModule._prepend_promptJs+11&$ &111sNc <[U5R[RLaV[R [U5R S35 U(aUR X5nUR"U40UD6$[[U5R S35e)a Preprocesses the input texts and returns a dictionary of preprocessed features. Args: inputs (list[SingleInput | PairInput]): List of inputs to preprocess. prompt (str | None): Optional prompt to prepend to text inputs. **kwargs: Additional keyword arguments for preprocessing, e.g. ``task``. Returns: dict[str, torch.Tensor | Any]: Dictionary containing preprocessed features, e.g. ``{"input_ids": ..., "attention_mask": ...}``, depending on what keys the module's forward method expects. zn overrides `tokenize` instead of `preprocess`. `tokenize` is deprecated, please override `preprocess` instead.z( must implement the `preprocess` method.)typetokenizer logger warning_once__name__rNotImplementedError)rrrkwargss r preprocessInputModule.preprocessOs* :  k&:&: :   :&&'(RR --f===262 2!T$Z%8%8$99a"bccrc R[RS5 UR"U40UD6$)a .. deprecated:: `tokenize` is deprecated. Use `preprocess` instead. Tokenizes the input texts and returns a dictionary of tokenized features. Args: texts (list[str]): List of input texts to tokenize. **kwargs: Additional keyword arguments for tokenization, e.g. ``task``. Returns: dict[str, torch.Tensor | Any]: Dictionary containing tokenized features, e.g. ``{"input_ids": ..., "attention_mask": ...}`` zEThe `tokenize` method is deprecated, please use `preprocess` instead.)r"r#r')rtextsr&s rr!InputModule.tokenizens+  S u///rc [US5(dg[UR[5(aURR"U40UD6 g[UR[ 5(aURR "U40UD6 g)z Saves the tokenizer to the specified output path. Args: output_path (str): Path to save the tokenizer. **kwargs: Additional keyword arguments for saving the tokenizer. Returns: None rN)hasattr isinstancerrsave_pretrainedrsave)r output_pathr&s rsave_tokenizerInputModule.save_tokenizersot[))  dnn&= > > NN * *; A& A  2 2 NN   6v 6rr)returnzlist[Modality])r list[str]rstrr4r5)N)rzlist[SingleInput | PairInput]rz str | Noner4dict[str, torch.Tensor | Any])r*r5r4r7)r1r6r4None)r$ __module__ __qualname____firstlineno____doc__r__annotations__propertyr staticmethodrr'r!r2__static_attributes__rrrr r s{)VL$22 22"d-dd ' d>0(rr ) __future__rtypingrtorch tokenizersr$transformers.tokenization_utils_basertransformers.utilsrtransformers_logging)sentence_transformers.base.modality_typesrr r )sentence_transformers.base.modules.moduler get_loggerr$r"r rrrrKs@" H>VV<  ( ( 2C&Cr