j!SrSSKrSSKJr SSKJrJrJrJr SSK J r J r J r SSK Jr SSKJr SSKJrJr \R("\5r"S S \5r"S S \5r"S S\5r"SS\\5r"SS\5r"SS\5rg)zEngine options for VLM inference. This module defines engine-specific configuration options that are independent of model specifications and prompts. N)Enum)AnyDictLiteralOptional)AnyUrlFieldmodel_validator)AcceleratorDevice)default_compile_model)BaseVlmEngineOptions VlmEngineTypecn\rSrSr%Sr\R r\\R \ S'\ "SSS9r \ \ S'Sr g ) AutoInlineVlmEngineOptionszOptions for auto-selecting the best local inference engine. Automatically selects the best available local engine based on: - Platform (macOS -> MLX, Linux/Windows -> Transformers/VLLM) - Available hardware (CUDA, MPS, CPU) - Model support engine_typeFz=Prefer VLLM over Transformers when both are available on CUDAdefault description prefer_vllmN)__name__ __module__ __qualname____firstlineno____doc__r AUTO_INLINErr__annotations__r rbool__static_attributes__ra/home/mande/repo/quber/.venv/lib/python3.13/site-packages/docling/datamodel/vlm_engine_options.pyrrs<7D6O6OK223OSKr!rc"\rSrSr%Sr\R r\\R \ S'\ "SSS9r \ \ \ S'\ "SS S9r\\ S '\ "S S S9r\\ S '\ "SSS9r\\ S'\ "SSS9r\ \\ S'\ "SSS9r\\ S'\ "SSS9r\\ S'\ "\SS9r\\ S'Srg)TransformersVlmEngineOptions5z6Options for HuggingFace Transformers inference engine.rN%Device to use (auto-detected if None)rdeviceTz0Load model in 8-bit precision using bitsandbytes load_in_8bitg@z%Threshold for LLM.int8() quantizationllm_int8_thresholdFz"Whether the model is pre-quantized quantizedz+PyTorch dtype (e.g., 'float16', 'bfloat16') torch_dtype.Allow execution of custom code from model repotrust_remote_codez&Enable key-value caching for attention use_kv_cachezIWhether to compile the model with torch.compile() for better performance.default_factoryr compile_modelr)rrrrrr TRANSFORMERSrrrr r'rr r(rr)floatr*r+strr-r.r r1r rr!r"r$r$5s@7D7Q7QK334Q*/"I+FH& '"TL$!&!H!#GIt"'"O"K#$#St"JL$ -_M4r!r$cn\rSrSr%Sr\R r\\R \ S'\ "SSS9r \ \ S'Sr g ) MlxVlmEngineOptionsazr?r@r rr!r"r:r:ps% D D DI-)r!r:c\rSrSr%Sr\R r\\R \ S'\ "SSS9r \ \ \ S'\ "SS S9r\\ S '\ "S S S9r\\ S '\ "SSS9r\\ S'\ "\R*SS9r\\ S'\ "SSS9r\\ S'Srg)VllmVlmEngineOptionszrGrIr4r rr!r"rBrBsF/rms //33C< "!5,$#7$X.'*T'*T'/'^...r!