o 0iL@sddlZddlmZmZddlZddlZddlmZddl m Z ddl m Z m Z ddlmZmZmZmZmZmZmZerFddlmmZdd ed ed efd dZddee deejfddZddeee e fdeejfddZ!dS)N)OptionalUnion)AcceleratorState)CUDA_DISTRIBUTED_TYPES)DistributedTypeRNGType)is_hpu_availableis_mlu_availableis_musa_availableis_npu_availableis_sdaa_availableis_torch_xla_availableis_xpu_availableFseeddevice_specific deterministiccCs|r|tj7}t|tj|t|tr"tj |n8t r,tj |n.t r6tj |n$tr@tj |ntrJtj |ntrTtj |ntj |trbt||rktddSdS)a Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. Args: seed (`int`): The seed to set. device_specific (`bool`, *optional*, defaults to `False`): Whether to differ the seed on each device slightly with `self.process_index`. deterministic (`bool`, *optional*, defaults to `False`): Whether to use deterministic algorithms where available. Can slow down training. TN)r process_indexrandomrnptorch manual_seedrxpumanual_seed_allr npur mlursdaar musar hpucudarxm set_rng_stateuse_deterministic_algorithms)rrrr$Y/sda-disk/www/egybert/egybert_env/lib/python3.10/site-packages/accelerate/utils/random.pyset_seed's.      r&rng_type generatorcCs8|tjkr t}n|tjkrtj}n|tjkr)ts!Jdtt }n}|tj kr;t s5Jdtj }nk|tj krMtsGJdtj}nY|tjkr_tsYJdtj}nG|tjkrqtskJdtj}n5|tjkrts}Jdtj}n#|tjkrtsJdtj}n|tjkr|dusJd|}t}|jt jkr|!t "}t #|gt $|%}nJ|jt&vs|jt j'ks|jt j(ks|jt j)ks|jt j*ks|jt j+ks|jt j,kr|!|j-}tj./|d |%}n|jt j0krtj./|d |tjkrt1|dS|tjkr*tj1|dS|tj kr8tj 1|dS|tj krFtj1|dS|tjkrTtj1|dS|tjkrbtj1|dS|tjkrptj1|dS|tjkr~tj1|dS|tjkrt 1|2dS|tjkr|3|dSdS) Nz8Can't synchronize XLA seeds as torch_xla is unavailable.z;Can't synchronize NPU seeds on an environment without NPUs.z;Can't synchronize MLU seeds on an environment without MLUs.z=Can't synchronize SDAA seeds on an environment without SDAAs.z=Can't synchronize MUSA seeds on an environment without MUSAs.z;Can't synchronize XPU seeds on an environment without XPUs.z;Can't synchronize HPU seeds on an environment without HPUs.z)Need a generator to synchronize its seed.r)4r TORCHr get_rng_stateCUDAr XLArtensorr!NPUr rMLUr rSDAArrMUSAr rXPUrrHPUr r GENERATOR get_staterdistributed_typerto xla_devicecollective_broadcast mark_stepcpur MULTI_MLU MULTI_SDAA MULTI_MUSA MULTI_NPU MULTI_XPU MULTI_HPUdevice distributed broadcast MULTI_CPUr"item set_state)r'r( rng_statestater$r$r%synchronize_rng_stateNs                                       rJ rng_typescCs|D] }tt||dqdS)N)r()rJr )rKr(r'r$r$r%synchronize_rng_statessrL)FF)NN)N)"rtypingrrnumpyrrrIr constantsr dataclassesrr importsr r r r rrrtorch_xla.core.xla_modelcore xla_modelr!intboolr& GeneratorrJliststrrLr$r$r$r%s  $ '*L