o 0if@sdZddlZddlZddlZddlZddlZddlmZddlZddl m Z ddl m Z m Z mZmZmZmZmZmZmZddlmZdd d Zd d Zd edefddZ   ddeededeefddZdefddZdS)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)Optional)version) is_cuda_availableis_hpu_availableis_ipex_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_sdaa_availableis_xpu_available)compare_versionsFcCs|rttrtjdStrtjdStr$tj dSt r.tj dSt r8tj dStddrDtjdStrNtjdStrT dSdS)z Clears the device cache by calling `torch.{backend}.empty_cache`. Can also run `gc.collect()`, but do note that this is a *considerable* slowdown and should be used sparingly. z2.0) min_versionN)gccollectr torchxpu empty_cachermlur sdaar musar npur mpsrcudargarbage_collectionrY/sda-disk/www/egybert/egybert_env/lib/python3.10/site-packages/accelerate/utils/memory.pyclear_device_cache,s& rcGs:t|ts t|}tt|D]}d||<qtdd|S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` NTr) isinstancelistrangelenr)objectsirrrrelease_memoryGs   r& exceptionreturncs:gd}ttrtjdkrtfdd|DSdS)z Checks if `exception` relates to CUDA out-of-memory, XPU out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )z out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryz1FATAL ERROR :: MODULE:PT_DEVMEM Allocation failedrc3s|] }|jdvVqdS)rN)args).0errr'rr tsz+should_reduce_batch_size..F)r RuntimeErrorr#r)any)r' _statementsrr,rshould_reduce_batch_sizeesr1functionstarting_batch_sizereduce_batch_size_fncsBdur tjt|dS|durfddfdd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is multiplied by 0.9 and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` N)r4cstdS)Ng?)intr) batch_sizerrr5s z8find_executable_batch_size..reduce_batch_size_fnc stddttj}t|t|dkr>dddt|dd|ddD}t dj dj d |d  d krGt d z g|Ri|WSt ys}zt |rhtddnWYd}~nd}~wwq?) NTrrz, cSsg|] \}}|d|qS)=r)r*argvaluerrr szAfind_executable_batch_size..decorator..zBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`rz-No executable batch size found, reached zero.)rr!inspect signature parameterskeysr#joinzip TypeError__name__r. Exceptionr1)r)kwargsparamsarg_strer7r3r5rr decorators2 *   z-find_executable_batch_size..decorator) functoolspartialfind_executable_batch_size)r3r4r5rKrrJrrNxs rN device_indexcCsttjjtdjkr z tj|dWStyYnwtr>tt j d}t |ddr>ddl m}||dSt dtj|S)Nz2.6rintel_extension_for_pytorchz>=z2.5) mem_get_infozThe XPU `mem_get_info` API is available in IPEX version >=2.5 or PyTorch >=2.6. The current returned available memory is incorrect. Please consider upgrading your IPEX or PyTorch version.)rparser __version__releaserrQrEr importlibmetadatarintel_extension_for_pytorch.xpuwarningswarnmax_memory_allocated)rO ipex_versionrQrrrget_xpu_available_memorys     r\)F)Nr2N) __doc__rLrrUr=rXtypingrr packagingrimportsrrrrr r r r r versionsrrr&rEboolr1callabler6rNr\rrrrs2  ,  D