docs: update the transformers and API codes
Browse files
    	
        README.md
    CHANGED
    
    | @@ -26,13 +26,15 @@ Embeddings produced by `jina-embeddings-v4` serve as the backbone for neural inf | |
| 26 |  | 
| 27 |  | 
| 28 | 
             
            Built based on [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), `jina-embeddings-v4` has the following features:
         | 
| 29 | 
            -
             | 
|  | |
| 30 | 
             
            - **Multilingual support** (20+ languages) and compatibility with a wide range of domains, including technical and visually complex documents.
         | 
| 31 | 
             
            - **Task-specific adapters** for retrieval, text matching, and code-related tasks, which can be selected at inference time.
         | 
| 32 | 
             
            - **Flexible embedding size**: dense embeddings are 2048 dimensions by default but can be truncated to as low as 128 with minimal performance loss.
         | 
| 33 |  | 
| 34 |  | 
| 35 | 
             
            Summary of features:
         | 
|  | |
| 36 | 
             
            | Feature   | Jina Embeddings V4   |
         | 
| 37 | 
             
            |------------|------------|
         | 
| 38 | 
             
            | Base Model | Qwen2.5-VL-3B-Instruct |
         | 
| @@ -42,8 +44,8 @@ Summary of features: | |
| 42 | 
             
            | Single-Vector Dimension | 2048 |
         | 
| 43 | 
             
            | Multi-Vector Dimension | 128 |
         | 
| 44 | 
             
            | Matryoshka dimensions | 128, 256, 512, 1024, 2048 |
         | 
| 45 | 
            -
            | Attention Mechanism | FlashAttention2 |
         | 
| 46 | 
             
            | Pooling Strategy | Mean pooling |
         | 
|  | |
| 47 |  | 
| 48 |  | 
| 49 |  | 
| @@ -58,6 +60,7 @@ Please refer to our [technical report of jina-embeddings-v4](https://puginarug.c | |
| 58 | 
             
              <summary>Requirements</a></summary>
         | 
| 59 |  | 
| 60 | 
             
            The following Python packages are required:
         | 
|  | |
| 61 | 
             
            - `transformers>=4.52.0`
         | 
| 62 | 
             
            - `torch>=2.6.0`
         | 
| 63 | 
             
            - `peft>=0.15.2`
         | 
| @@ -68,25 +71,21 @@ The following Python packages are required: | |
| 68 | 
             
            - **flash-attention**: Installing [flash-attention](https://github.com/Dao-AILab/flash-attention) is recommended for improved inference speed and efficiency, but not mandatory.
         | 
| 69 | 
             
            - **sentence-transformers**: If you want to use the model via the `sentence-transformers` interface, install this package as well.
         | 
| 70 |  | 
| 71 | 
            -
             | 
| 72 | 
             
            </details>
         | 
| 73 |  | 
| 74 |  | 
| 75 | 
             
            <details>
         | 
| 76 | 
            -
              <summary>via  | 
|  | |
| 77 |  | 
| 78 | 
            -
            Needs to be adjusted for V4
         | 
| 79 | 
             
            ```bash
         | 
| 80 | 
             
            curl https://api.jina.ai/v1/embeddings \
         | 
| 81 | 
             
              -H "Content-Type: application/json" \
         | 
| 82 | 
            -
              -H "Authorization: Bearer  | 
| 83 | 
             
              -d @- <<EOFEOF
         | 
| 84 | 
             
              {
         | 
| 85 | 
             
                "model": "jina-embeddings-v4",
         | 
| 86 | 
            -
                " | 
| 87 | 
            -
                "task": "retrieval.query",
         | 
| 88 | 
            -
                "normalized": true,
         | 
| 89 | 
            -
                "embedding_type": "float",
         | 
| 90 | 
             
                "input": [
         | 
| 91 | 
             
                    {
         | 
| 92 | 
             
                        "text": "غروب جميل على الشاطئ"
         | 
| @@ -136,37 +135,41 @@ EOFEOF | |
| 136 |  | 
| 137 | 
             
            ```python
         | 
| 138 | 
             
            # !pip install transformers>=4.52.0 torch>=2.6.0 peft>=0.15.2 torchvision pillow
         | 
| 139 | 
            -
            # !pip install | 
| 140 | 
             
            from transformers import AutoModel
         | 
|  | |
| 141 |  | 
| 142 | 
             
            # Initialize the model
         | 
| 143 | 
             
            model = AutoModel.from_pretrained("jinaai/jina-embeddings-v4", trust_remote_code=True)
         | 
|  | |
|  | |
|  | |
| 144 | 
             
            # ========================
         | 
| 145 | 
             
            # 1. Retrieval Task
         | 
| 146 | 
             
            # ========================
         | 
| 147 | 
             
            # Configure truncate_dim, max_length (for texts), max_pixels (for images), vector_type, batch_size in the encode function if needed
         | 
| 148 |  | 
| 149 | 
             
            # Encode query
         | 
| 150 | 
            -
             | 
| 151 | 
             
                texts=["Overview of climate change impacts on coastal cities"],
         | 
| 152 | 
             
                task="retrieval",
         | 
| 153 | 
             
                prompt_name="query",
         | 
| 154 | 
            -
            ) | 
| 155 |  | 
| 156 | 
             
            # Encode passage (text)
         | 
| 157 | 
            -
             | 
| 158 | 
             
                texts=[
         | 
| 159 | 
             
                    "Climate change has led to rising sea levels, increased frequency of extreme weather events..."
         | 
| 160 | 
             
                ],
         | 
| 161 | 
             
                task="retrieval",
         | 
| 162 | 
             
                prompt_name="passage",
         | 
| 163 | 
            -
            ) | 
| 164 |  | 
| 165 | 
             
            # Encode image/document
         | 
| 166 | 
            -
             | 
| 167 | 
             
                images=["https://i.ibb.co/nQNGqL0/beach1.jpg"],
         | 
| 168 | 
             
                task="retrieval",
         | 
| 169 | 
            -
            ) | 
| 170 |  | 
| 171 | 
             
            # ========================
         | 
| 172 | 
             
            # 2. Text Matching Task
         | 
| @@ -183,25 +186,43 @@ texts = [ | |
| 183 | 
             
                "해변 위로 아름다운 일몰",  # Korean
         | 
| 184 | 
             
            ]
         | 
| 185 |  | 
| 186 | 
            -
            text_embeddings = model. | 
| 187 |  | 
| 188 | 
             
            # ========================
         | 
| 189 | 
             
            # 3. Code Understanding Task
         | 
| 190 | 
             
            # ========================
         | 
| 191 |  | 
| 192 | 
             
            # Encode query
         | 
| 193 | 
            -
            query_embedding = model. | 
| 194 | 
             
                texts=["Find a function that prints a greeting message to the console"],
         | 
| 195 | 
             
                task="code",
         | 
| 196 | 
             
                prompt_name="query",
         | 
| 197 | 
             
            )
         | 
| 198 |  | 
| 199 | 
             
            # Encode code
         | 
| 200 | 
            -
            code_embeddings = model. | 
| 201 | 
             
                texts=["def hello_world():\n    print('Hello, World!')"],
         | 
| 202 | 
             
                task="code",
         | 
| 203 | 
             
                prompt_name="passage",
         | 
| 204 | 
             
            )
         | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
| 205 | 
             
            ```
         | 
| 206 | 
             
            </details>
         | 
| 207 |  | 
|  | |
| 26 |  | 
| 27 |  | 
| 28 | 
             
            Built based on [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), `jina-embeddings-v4` has the following features:
         | 
| 29 | 
            +
             | 
| 30 | 
            +
            - **Unified embeddings** for text, images, and visual documents, supporting both dense (single-vector) and late-interaction (multi-vector) retrieval.
         | 
| 31 | 
             
            - **Multilingual support** (20+ languages) and compatibility with a wide range of domains, including technical and visually complex documents.
         | 
| 32 | 
             
            - **Task-specific adapters** for retrieval, text matching, and code-related tasks, which can be selected at inference time.
         | 
| 33 | 
             
            - **Flexible embedding size**: dense embeddings are 2048 dimensions by default but can be truncated to as low as 128 with minimal performance loss.
         | 
| 34 |  | 
| 35 |  | 
| 36 | 
             
            Summary of features:
         | 
| 37 | 
            +
             | 
| 38 | 
             
            | Feature   | Jina Embeddings V4   |
         | 
| 39 | 
             
            |------------|------------|
         | 
| 40 | 
             
            | Base Model | Qwen2.5-VL-3B-Instruct |
         | 
|  | |
| 44 | 
             
            | Single-Vector Dimension | 2048 |
         | 
| 45 | 
             
            | Multi-Vector Dimension | 128 |
         | 
| 46 | 
             
            | Matryoshka dimensions | 128, 256, 512, 1024, 2048 |
         | 
|  | |
| 47 | 
             
            | Pooling Strategy | Mean pooling |
         | 
| 48 | 
            +
            | Attention Mechanism | FlashAttention2 |
         | 
| 49 |  | 
| 50 |  | 
| 51 |  | 
|  | |
| 60 | 
             
              <summary>Requirements</a></summary>
         | 
| 61 |  | 
| 62 | 
             
            The following Python packages are required:
         | 
| 63 | 
            +
             | 
| 64 | 
             
            - `transformers>=4.52.0`
         | 
| 65 | 
             
            - `torch>=2.6.0`
         | 
| 66 | 
             
            - `peft>=0.15.2`
         | 
|  | |
| 71 | 
             
            - **flash-attention**: Installing [flash-attention](https://github.com/Dao-AILab/flash-attention) is recommended for improved inference speed and efficiency, but not mandatory.
         | 
| 72 | 
             
            - **sentence-transformers**: If you want to use the model via the `sentence-transformers` interface, install this package as well.
         | 
| 73 |  | 
|  | |
| 74 | 
             
            </details>
         | 
| 75 |  | 
| 76 |  | 
| 77 | 
             
            <details>
         | 
| 78 | 
            +
              <summary>via <a href="https://jina.ai/embeddings/">Jina AI Embeddings API</a></summary>
         | 
| 79 | 
            +
             | 
| 80 |  | 
|  | |
| 81 | 
             
            ```bash
         | 
| 82 | 
             
            curl https://api.jina.ai/v1/embeddings \
         | 
| 83 | 
             
              -H "Content-Type: application/json" \
         | 
| 84 | 
            +
              -H "Authorization: Bearer $JINA_AI_API_TOKEN" \
         | 
| 85 | 
             
              -d @- <<EOFEOF
         | 
| 86 | 
             
              {
         | 
| 87 | 
             
                "model": "jina-embeddings-v4",
         | 
| 88 | 
            +
                "task": "text-matching",
         | 
|  | |
|  | |
|  | |
| 89 | 
             
                "input": [
         | 
| 90 | 
             
                    {
         | 
| 91 | 
             
                        "text": "غروب جميل على الشاطئ"
         | 
|  | |
| 135 |  | 
| 136 | 
             
            ```python
         | 
| 137 | 
             
            # !pip install transformers>=4.52.0 torch>=2.6.0 peft>=0.15.2 torchvision pillow
         | 
| 138 | 
            +
            # !pip install
         | 
| 139 | 
             
            from transformers import AutoModel
         | 
| 140 | 
            +
            import torch
         | 
| 141 |  | 
| 142 | 
             
            # Initialize the model
         | 
| 143 | 
             
            model = AutoModel.from_pretrained("jinaai/jina-embeddings-v4", trust_remote_code=True)
         | 
| 144 | 
            +
             | 
| 145 | 
            +
            model.to("cuda")
         | 
| 146 | 
            +
             | 
| 147 | 
             
            # ========================
         | 
| 148 | 
             
            # 1. Retrieval Task
         | 
| 149 | 
             
            # ========================
         | 
| 150 | 
             
            # Configure truncate_dim, max_length (for texts), max_pixels (for images), vector_type, batch_size in the encode function if needed
         | 
| 151 |  | 
| 152 | 
             
            # Encode query
         | 
| 153 | 
            +
            query_embeddings = model.encode_text(
         | 
| 154 | 
             
                texts=["Overview of climate change impacts on coastal cities"],
         | 
| 155 | 
             
                task="retrieval",
         | 
| 156 | 
             
                prompt_name="query",
         | 
| 157 | 
            +
            )
         | 
| 158 |  | 
| 159 | 
             
            # Encode passage (text)
         | 
| 160 | 
            +
            passage_embeddings = model.encode_text(
         | 
| 161 | 
             
                texts=[
         | 
| 162 | 
             
                    "Climate change has led to rising sea levels, increased frequency of extreme weather events..."
         | 
| 163 | 
             
                ],
         | 
| 164 | 
             
                task="retrieval",
         | 
| 165 | 
             
                prompt_name="passage",
         | 
| 166 | 
            +
            )
         | 
| 167 |  | 
| 168 | 
             
            # Encode image/document
         | 
| 169 | 
            +
            image_embeddings = model.encode_image(
         | 
| 170 | 
             
                images=["https://i.ibb.co/nQNGqL0/beach1.jpg"],
         | 
| 171 | 
             
                task="retrieval",
         | 
| 172 | 
            +
            )
         | 
| 173 |  | 
| 174 | 
             
            # ========================
         | 
| 175 | 
             
            # 2. Text Matching Task
         | 
|  | |
| 186 | 
             
                "해변 위로 아름다운 일몰",  # Korean
         | 
| 187 | 
             
            ]
         | 
| 188 |  | 
| 189 | 
            +
            text_embeddings = model.encode_text(texts=texts, task="text-matching")
         | 
| 190 |  | 
| 191 | 
             
            # ========================
         | 
| 192 | 
             
            # 3. Code Understanding Task
         | 
| 193 | 
             
            # ========================
         | 
| 194 |  | 
| 195 | 
             
            # Encode query
         | 
| 196 | 
            +
            query_embedding = model.encode_text(
         | 
| 197 | 
             
                texts=["Find a function that prints a greeting message to the console"],
         | 
| 198 | 
             
                task="code",
         | 
| 199 | 
             
                prompt_name="query",
         | 
| 200 | 
             
            )
         | 
| 201 |  | 
| 202 | 
             
            # Encode code
         | 
| 203 | 
            +
            code_embeddings = model.encode_text(
         | 
| 204 | 
             
                texts=["def hello_world():\n    print('Hello, World!')"],
         | 
| 205 | 
             
                task="code",
         | 
| 206 | 
             
                prompt_name="passage",
         | 
| 207 | 
             
            )
         | 
| 208 | 
            +
             | 
| 209 | 
            +
            # ========================
         | 
| 210 | 
            +
            # 4. Use multivectors
         | 
| 211 | 
            +
            # ========================
         | 
| 212 | 
            +
             | 
| 213 | 
            +
            multivector_embeddings = model.encode_text(
         | 
| 214 | 
            +
                texts=texts,
         | 
| 215 | 
            +
                task="retrieval",
         | 
| 216 | 
            +
                prompt_name="query",
         | 
| 217 | 
            +
                return_multivector=True,
         | 
| 218 | 
            +
            )
         | 
| 219 | 
            +
             | 
| 220 | 
            +
            images = ["https://i.ibb.co/nQNGqL0/beach1.jpg", "https://i.ibb.co/r5w8hG8/beach2.jpg"]
         | 
| 221 | 
            +
            multivector_image_embeddings = model.encode_image(
         | 
| 222 | 
            +
                images=images,
         | 
| 223 | 
            +
                task="retrieval",
         | 
| 224 | 
            +
                return_multivector=True,
         | 
| 225 | 
            +
            )
         | 
| 226 | 
             
            ```
         | 
| 227 | 
             
            </details>
         | 
| 228 |  | 

