o i@sUddlmZddlmZmZmZmZddlmZddl m Z ddl m Z m Z ddlmZe r._is_valid_imagerc3|]}|VqdSNr.0r$r%rr 9z'validate_image_pairs..c3s<|]}t|tot|dkotfdd|DVqdS)rc3r&r'rr(r*rrr+>r,z1validate_image_pairs...N)r"rlenall)r) image_pairr*rrr+;s  )r"rr-r. ValueError)r error_messagerr*rvalidate_image_pairs-s " r2c seZdZdZdZdZdZdZfddZdddZ e dd e d e d e d eefddZe dd ee d e d e d eeefddZ dd ee e Bd e d e d eeeeeBffdd ZdddZddZdd eefddZZS)KeypointMatchingPipelinez Keypoint matching pipeline using any `AutoModelForKeypointMatching`. This pipeline matches keypoints between two images. FTcs tj|i|t|ddS)Nvision)super__init__r )selfargskwargs __class__rrr6Osz!KeypointMatchingPipeline.__init__NcCs2i}|dur ||d<i}|dur||d<|i|fS)Ntimeout thresholdr)r7r=r<preprocess_paramspostprocess_paramsrrr_sanitize_parametersSs z-KeypointMatchingPipeline._sanitize_parametersinputsr=r9r!cKdSr'rr7rBr=r9rrr__call__\z!KeypointMatchingPipeline.__call__cKrCr'rrDrrrrE_rFc sH|durtdt|}tj|fd|i|}t|dkr"|dS|S)a Find matches between keypoints in two images. Args: inputs (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single pair of images or a batch of image pairs, which must then be passed as a string. Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL images. threshold (`float`, *optional*, defaults to 0.0): The threshold to use for keypoint matching. Keypoints matched with a lower matching score will be filtered out. A value of 0 means that all matched keypoints will be returned. kwargs: `timeout (`float`, *optional*, defaults to None)` The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: Union[list[Match], list[list[Match]]]: A list of matches or a list if a single image pair is provided, or of lists of matches if a batch of image pairs is provided. Each match is a dictionary containing the following keys: - **keypoint_image_0** (`Keypoint`): The keypoint in the first image (x, y coordinates). - **keypoint_image_1** (`Keypoint`): The keypoint in the second image (x, y coordinates). - **score** (`float`): The matching score between the two keypoints. NzFCannot call the keypoint-matching pipeline without an inputs argument!r=r r)r0r2r5rEr-)r7rBr=r9formatted_inputsoutputsr:rrrEbs' csHfdd|D}|j|dd}||j}dd|D}||d}|S)Ncsg|]}t|dqS)r<rr(rIrr sz7KeypointMatchingPipeline.preprocess..pt)r return_tensorscSsg|]}|jqSr)sizer(rrrrJs) model_inputs target_sizes)image_processortodtype)r7r r<rNrOpreprocess_outputsrrIr preprocesss   z#KeypointMatchingPipeline.preprocesscCs,|d}|jdi|}||dgd}|S)NrNrO) model_outputsrOr)model)r7rSrNrUforward_outputsrrr_forwardsz!KeypointMatchingPipeline._forwardc Cs|d}|d}|jj|||d}|d}g}t|d|d|dD]-\}}} t|d|dd }t|d|dd }|t||| d q#t|d d d d}|S)NrUrO)rOr=r keypoints0 keypoints1matching_scoresr )rr)rrrcSs|dS)Nrr)rrrrsz6KeypointMatchingPipeline.postprocess..T)keyreverse)rPpost_process_keypoint_matchingzipritemappendrsorted) r7rWr=rUrOpostprocess_outputs pair_resultkp_0kp_1rrrr postprocesss"z$KeypointMatchingPipeline.postprocess)NN)rAr')rrr__doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerr6r@rrrrlistrrErTrXrh __classcell__rrr:rr3Es2   "*  /r3N)collections.abcrtypingrrrrtyping_extensionsr image_utilsr utilsr r baser PILrrr#rrrrr2r3rrrrs