o iN @sRddlmZddlmZddlmZmZmZeedddd Gd d d eZd S) )Any)add_end_docstrings) GenericTensorPipelinebuild_pipeline_init_argsTF) has_tokenizersupports_binary_outputa tokenize_kwargs (`dict`, *optional*): Additional dictionary of keyword arguments passed along to the tokenizer. return_tensors (`bool`, *optional*): If `True`, returns a tensor according to the specified framework, otherwise returns a list.cseZdZdZdZdZdZdZdddZde e e ffdd Z d d Z dd d Zde ee BdedeeeBffdd ZZS)FeatureExtractionPipelinea Feature extraction pipeline uses no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks. Example: ```python >>> from transformers import pipeline >>> extractor = pipeline(model="google-bert/bert-base-uncased", task="feature-extraction") >>> result = extractor("This is a simple test.", return_tensors=True) >>> result.shape # This is a tensor of shape [1, sequence_length, hidden_dimension] representing the input string. torch.Size([1, 8, 768]) ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This feature extraction pipeline can currently be loaded from [`pipeline`] using the task identifier: `"feature-extraction"`. All models may be used for this pipeline. See a list of all models, including community-contributed models on [huggingface.co/models](https://huggingface.co/models). FTNcKsN|duri}|durd|vrtd||d<|}i}|dur"||d<|i|fS)N truncationz\truncation parameter defined twice (given as keyword argument as well as in tokenize_kwargs)return_tensors) ValueError)selfr tokenize_kwargsr kwargspreprocess_paramspostprocess_paramsrk/sda-disk/www/egybert/egybert_env/lib/python3.10/site-packages/transformers/pipelines/feature_extraction.py_sanitize_parameters-s z.FeatureExtractionPipeline._sanitize_parametersreturncKs|j|fddi|}|S)Nr pt) tokenizer)rinputsr model_inputsrrr preprocess@sz$FeatureExtractionPipeline.preprocesscCs|jdi|}|S)Nr)model)rr model_outputsrrr_forwardDsz"FeatureExtractionPipeline._forwardcCs|r|dS|dS)Nr)tolist)rrr rrr postprocessHs z%FeatureExtractionPipeline.postprocessargsrcstj|i|S)a Extract the features of the input(s) text. Args: args (`str` or `list[str]`): One or several texts (or one list of texts) to get the features of. Return: A nested list of `float`: The features computed by the model. )super__call__)rr"r __class__rrr$Ns z"FeatureExtractionPipeline.__call__)NNN)F)__name__ __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrdictstrrrrr!listrr$ __classcell__rrr%rr s   2r N) typingrutilsrbaserrrr rrrrs