o 0iS @sLddlmZddlmZdZddZd ddZd d Zed kr$edSdS) )CustomArgumentParser)merge_fsdp_weightsa!Utility to merge the weights from multiple FSDP checkpoints into a single combined checkpoint. Should be used if `SHARDED_STATE_DICT` was used for the model. Weights will be saved to `{output_path}`. This is a CPU-bound process and requires enough RAM to load the entire model state dict.cCst|j|j|j |jdSN)rcheckpoint_directory output_pathunsafe_serializationremove_checkpoint_dir)argsr [/sda-disk/www/egybert/egybert_env/lib/python3.10/site-packages/accelerate/commands/merge.py merge_commandsr NcCs~|dur |jdtd}nttd}|jdtdd|jdtdd|jdd d d d |jd d dd d|dur=|jtd|S)Nz merge-weights) descriptionrz5A directory containing sharded weights saved by FSDP.)typehelprzHThe path to save the merged weights. Defaults to the current directory. z--unsafe_serialization store_trueFzZWhether to save the merged weights as `.bin` rather than `.safetensors` (not recommended).)actiondefaultrz--remove_checkpoint_dirz9Whether to remove the checkpoint directory after merging.)rrr)func) add_parserr r add_argumentstr set_defaultsr ) subparsersparserr r r merge_command_parser s0  rcCst}|}t|dSr)r parse_argsr )rr r r r main>s r__main__r) accelerate.commands.utilsraccelerate.utilsrr r rr__name__r r r r s