Ei
8 S SK r S SKJr S SKJr S SKrS SKrS SKJ r SSK
J r J r J
r
JrJrJrJrJrJrJrJrJrJrJrJrJrJrJ r J r J r J r \" SS5 ( a S S K J!r! OS S K"J!r! \ " 5 ( a S SK#J$s J%r& SS
K'J(r( SS K)J*r* \(" \+5 r, SS \-S
\.\/ S\.S\.S\.S\0S\0S\\! S\1S\14S jjr2 S S jr3S S\0S\14S jjr4S S\04S jjr5g) N)Path)Optional)
load_model )
MODEL_NAMEOPTIMIZER_NAMERNG_STATE_NAMESAFE_MODEL_NAMESAFE_WEIGHTS_NAME SAMPLER_NAME SCALER_NAMESCHEDULER_NAME WEIGHTS_NAMEget_pretty_nameis_cuda_availableis_hpu_availableis_mlu_availableis_musa_availableis_neuron_availableis_sdaa_availableis_torch_versionis_torch_xla_availableis_xpu_availableloadsavez>=z2.4.0)
GradScaler)
get_logger) PartialState
output_dir model_states
optimizers
schedulers dataloaders
process_indexstepscalersave_on_each_nodesafe_serializationc
[ U 5 n [ U5 Hg u pU ( d [ O[ n U
S: a U R SSU
S35 n U R U 5 n
[
XXS9 [ R SU
35 Mi [ U5 He u pUR 5 n U
S:X a [ S3O [ SU
S3nU R U5 n[
U UUSS9 [ R SU 35 Mg [ U5 Hf u n
nUR 5 n U
S:X a [ S3O [ SU
S3nU R U5 n[
U UUSS9 [ R S U 35 Mh [ U5 H u n
nU
S:X a [ S3O [ SU
S3nU R U5 nS
S K
JnJn [! UR" U5 ( a- UR% 5 n[! UU5 ( a [
UUUSS9 [' US S5 ( aF U
S:X a S
OSU
S3nU R U5 nUR 5 n [( R " U U5 [ R SU
SU 35 M UbT UR 5 n U R [* 5 n [( R " U U 5 [ R SU 35 0 n [, SU S3n UU S' [. R0 " 5 U S' [2 R. R5 5 U S' [( R6 " 5 U S' [9 5 ( a! [( R: R= 5 U S' [? 5 ( a" [( R@ R= 5 U S' Oa[C 5 ( a" [( RD R= 5 U S' O0[G 5 ( a! [( RH R= 5 U S' [K 5 ( a! [( RL R= 5 U S' [O 5 ( a! [( RP R= 5 U S ' [S 5 ( a! [( RT R= 5 U S ' [W 5 ( a [X R6 " 5 U S ' U R U 5 n [( R " U U 5 [ R S U 35 U $ ) a
Saves the current states of the models, optimizers, scaler, and RNG generators to a given directory.
If `safe_serialization` is `True`, models will be saved with `safetensors` while the rest are saved using native
`pickle`.
Args:
output_dir (`str` or `os.PathLike`):
The name of the folder to save all relevant weights and states.
model_states (`List[torch.nn.Module]`):
A list of model states
optimizers (`List[torch.optim.Optimizer]`):
A list of optimizer instances
schedulers (`List[torch.optim.lr_scheduler._LRScheduler]`):
A list of learning rate schedulers
dataloaders (`List[torch.utils.data.DataLoader]`):
A list of dataloader instances to save their sampler states
process_index (`int`):
The current process index in the Accelerator state
step (`int`):
The current step in the internal step tracker
scaler (`torch.amp.GradScaler`, *optional*):
An optional gradient scaler instance to save;
save_on_each_node (`bool`, *optional*):
Whether to save on every node, or only the main node.
safe_serialization (`bool`, *optional*, defaults to `True`):
Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
r ._)r' r( zModel weights saved in .binFzOptimizer state saved in zScheduler state saved in r IterableDatasetShardSeedableRandomSampleruse_stateful_dataloaderdl_state_dict.bindl_state_dict_z Sampler state for dataloader z
saved in z Gradient scaler state saved in .pklr% random_statenumpy_random_seedtorch_manual_seedtorch_xpu_manual_seedtorch_mlu_manual_seedtorch_sdaa_manual_seedtorch_musa_manual_seedtorch_hpu_manual_seedtorch_neuron_manual_seedtorch_cuda_manual_seedxm_seedzRandom states saved in )-r enumerater r replacejoinpathr loggerinfo
state_dictr r r data_loaderr. r/
isinstancedataset get_samplergetattrtorchr
r randomgetstatenp get_state
get_rng_stater xpuget_rng_state_allr mlur sdaar musar hpur neuronr cudar xm)!r r r! r" r# r$ r% r& r' r( istate weights_nameoutput_model_fileoptoptimizer_nameoutput_optimizer_file schedulerscheduler_nameoutput_scheduler_file
dataloader sampler_nameoutput_sampler_filer. r/ samplerdataloader_state_dict_name!output_dataloader_state_dict_filerD output_scaler_filestates states_nameoutput_states_files! U/home/mande/repo/quber/.venv/lib/python3.13/site-packages/accelerate/checkpointing.pysave_accelerator_statern ? sh X j!J l++=|CT q5'//q1X>L & / / = U9Jr -.?-@AB
, J' 45FN+40>BRRSTUSVVZ@[ * 3 3N C U)=Nchi /0E/FGH ( "*- 9$$&45FN+40>BRRSTUSVVZ@[ * 3 3N C U)=Nchi /0E/FGH . #;/
:01Q, t,|nAaSPT ? ? ,,.G'#899W 1EVkpq : 8% @ @@AQ)"$))"5"5"7F "'"5"5"7F */))*E*E*G&'*/))*E*E*G&' +0::+G+G+I'( +0::+G+G+I'(*/))*E*E*G&' -2\\-K-K-M)*+0::+G+G+I'( , , .y#,,[9 JJv)*
KK)*<)=>? c [ 5 n
US; a [ S5 eUc SnOUS:X a [ 5 R nUc 0 n[ U 5 n [ U5 H u pU S: a SU 3OSn
U R
[ U
S35 nUR 5 ( a [ X4S [ U5 0U D6 MZ U R
[ U
S
35 n[ XS 9nU R " U40 U D6 M [ R S 5 [ U5 HS u n nU S:X a [ S
3O [ SU S
3nU R
U5 n[ U4S
U0UD6nX+ R U5 MU [ R S5 [ U5 HO u n nU S:X a [" S
3O [" SU S
3nU R
U5 n[ U40 UD6nUR U5 MQ [ R S5 [ U5 H u n nU S:X a [$ S
3O [$ SU S
3nU R
U5 nSSKJnJn [- UR. U5 ( a; UR1 5 n [- U U 5 ( a UR3 [ U5 5 n [5 USS5 ( d M U S:X a SOSU S
3n U R
U 5 n U R 5 ( d M [ U 40 UD6nUR U5 M [ R S5 UbF U R
[6 5 n [ U 5 n!UR U!5 [ R S5 [ U R
[8 SU S35 5 n"SU"; a U"S U
S' [: R< " U"S 5 [> R: RA U"S 5 [B RD " U"S 5 [G 5 ( a" [B RH RK U"S 5 [M 5 ( a# [B RN RK U"S 5 O[Q 5 ( a# [B RR RK U"S 5 O[U 5 ( a# [B RV RK U"S 5 O[Y 5 ( a# [B RZ RK U"S! 5 OT[] 5 ( a# [B R^ RK U"S" 5 O"[B R` RK U"S# 5 [c 5 ( a [d RD " U"S$ 5 [ R S%5 U
$ ! [f a [ R S&5 U
$ f = f)'ax
Loads states of the models, optimizers, scaler, and RNG generators from a given directory.
Args:
input_dir (`str` or `os.PathLike`):
The name of the folder to load all relevant weights and states.
models (`List[torch.nn.Module]`):
A list of model instances
optimizers (`List[torch.optim.Optimizer]`):
A list of optimizer instances
schedulers (`List[torch.optim.lr_scheduler._LRScheduler]`):
A list of learning rate schedulers
process_index (`int`):
The current process index in the Accelerator state
scaler (`torch.amp.GradScaler`, *optional*):
An optional *GradScaler* instance to load
map_location (`str`, *optional*):
What device to load the optimizer state onto. Should be one of either "cpu" or "on_device".
load_kwargs (`dict`, *optional*):
Additional arguments that can be passed to the `load` function.
load_model_func_kwargs (`dict`, *optional*):
Additional arguments that can be passed to the model's `load_state_dict` method.
Returns:
`dict`: Contains the `Accelerator` attributes to override while loading the state.
)Ncpu on_devicezaUnsupported optimizer map location passed, please choose one of `None`, `'cpu'`, or `'on_device'`rq rr r r+ z .safetensorsdevicer, ) map_locationz%All model weights loaded successfullyru z(All optimizer states loaded successfullyz(All scheduler states loaded successfullyr r- r0 Fr1 r2 z1All dataloader sampler states loaded successfullyz$GradScaler state loaded successfullyr3 r% r4 r5 r6 r7 r8 r9 r: r; r< r= r> z%All random states loaded successfullyz Could not load random states)4dict TypeErrorr rt r r? rA r
existsr strr r load_state_dictrB rC r r r rE r. r/ rF rG rH set_samplerrI r
r rK setstaterM set_staterJ
set_rng_stater rP set_rng_state_allr rR r rS r rT r rU r rV rW r rX Exception)# input_dirmodelsr! r" r# r$ r& ru load_kwargsload_model_func_kwargsoverride_attributesrY modelendinginput_model_filerD r] r^ input_optimizer_fileoptimizer_stater` ra input_scheduler_filescheduler_staterc rd input_sampler_filer. r/ rf rg input_dataloader_state_dict_fileinput_scaler_file scaler_staterj s# rm load_accelerator_stater s N &55 o
$#~,, YI f% E1QCr$-- 0A& .VW " " $ $ u cs<7H cLb c ) 1 1ZL2M N .JJ ! !* G0F G & KK78 J'345FN+40>BRRSTUSVVZ@[ ( 1 1. A 3^,^R]^
%%o6 (
KK:; "*- 945FN+40>BRRSTUSVVZ@[ ( 1 1. A 3C{C!!/2 .
KK:;";/
:01Q, t,|nAaSPT ? ? ,,.G'#899$006H1IJ : 8% @ @@AQ)r s( ( 2 D'""$)))
H $( # #uut*u u u
u
u u
Z u u u~ G TO OD OU Uro