Clone of vidore/colpali-v1.3-hf

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,120 @@
+---
+library_name: transformers
+tags:
+- colpali
+license: gemma
+datasets:
+- vidore/colpali_train_set
+language:
+- en
+base_model:
+- vidore/colpaligemma-3b-pt-448-base
+pipeline_tag: visual-document-retrieval
+---
+
+> [!IMPORTANT]
+> This version of ColPali should be loaded with the `transformers 🤗` release, not with `colpali-engine`.
+> It was converted using the [`convert_colpali_weights_to_hf.py` script](https://github.com/tonywu71/transformers/blob/21c1309637aee97ca4fb8eb3b31830913a0f99a5/src/transformers/models/colpali/convert_colpali_weights_to_hf.py)
+> from the [`vidore/colpali-v1.3-merged`](https://huggingface.co/vidore/colpali-v1.3-merged) checkpoint.
+
+# ColPali: Visual Retriever based on PaliGemma-3B with ColBERT strategy
+
+ColPali is a model based on a novel model architecture and training strategy based on Vision Language Models (VLMs) to efficiently index documents from their visual features.
+It is a [PaliGemma-3B](https://huggingface.co/google/paligemma-3b-mix-448) extension that generates [ColBERT](https://arxiv.org/abs/2004.12832)- style multi-vector representations of text and images. 
+It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
+
+The HuggingFace `transformers` 🤗 implementation was contributed by Tony Wu ([@tonywu71](https://huggingface.co/tonywu71)) and Yoni Gozlan ([@yonigozlan](https://huggingface.co/yonigozlan)).
+
+<p align="center"><img width=800 src="https://github.com/illuin-tech/colpali/blob/main/assets/colpali_architecture.webp?raw=true"/></p>
+
+## Model Description
+
+Read the `transformers` 🤗 model card: https://huggingface.co/docs/transformers/en/model_doc/colpali.
+
+## Model Training
+
+### Dataset
+Our training dataset of 127,460 query-page pairs is comprised of train sets of openly available academic datasets (63%) and a synthetic dataset made up of pages from web-crawled PDF documents and augmented with VLM-generated (Claude-3 Sonnet) pseudo-questions (37%). 
+Our training set is fully English by design, enabling us to study zero-shot generalization to non-English languages. We explicitly verify no multi-page PDF document is used both [*ViDoRe*](https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d) and in the train set to prevent evaluation contamination. 
+A validation set is created with 2% of the samples to tune hyperparameters.
+
+*Note: Multilingual data is present in the pretraining corpus of the language model (Gemma-2B) and potentially occurs during PaliGemma-3B's multimodal training.*
+
+### Parameters
+
+All models are trained for 1 epoch on the train set. Unless specified otherwise, we train models in `bfloat16` format, use low-rank adapters ([LoRA](https://arxiv.org/abs/2106.09685)) 
+with `alpha=32`  and `r=32` on the transformer layers from the language model, 
+as well as the final randomly initialized projection layer, and use a `paged_adamw_8bit` optimizer. 
+We train on an 8 GPU setup with data parallelism, a learning rate of 5e-5 with linear decay with 2.5% warmup steps, and a batch size of 32.
+
+## Usage
+
+```python
+import torch
+from PIL import Image
+
+from transformers import ColPaliForRetrieval, ColPaliProcessor
+
+model_name = "vidore/colpali-v1.3-hf"
+
+model = ColPaliForRetrieval.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda:0",  # or "mps" if on Apple Silicon
+).eval()
+
+processor = ColPaliProcessor.from_pretrained(model_name)
+
+# Your inputs
+images = [
+    Image.new("RGB", (32, 32), color="white"),
+    Image.new("RGB", (16, 16), color="black"),
+]
+queries = [
+    "What is the organizational structure for our R&D department?",
+    "Can you provide a breakdown of last year’s financial performance?",
+]
+
+# Process the inputs
+batch_images = processor(images=images).to(model.device)
+batch_queries = processor(text=queries).to(model.device)
+
+# Forward pass
+with torch.no_grad():
+    image_embeddings = model(**batch_images)
+    query_embeddings = model(**batch_queries)
+
+# Score the queries against the images
+scores = processor.score_retrieval(query_embeddings.embeddings, image_embeddings.embeddings)
+```
+
+## Limitations
+
+ - **Focus**: The model primarily focuses on PDF-type documents and high-ressources languages, potentially limiting its generalization to other document types or less represented languages.
+ - **Support**: The model relies on multi-vector retreiving derived from the ColBERT late interaction mechanism, which may require engineering efforts to adapt to widely used vector retrieval frameworks that lack native multi-vector support.
+
+## License
+
+ColPali's vision language backbone model (PaliGemma) is under `gemma` license as specified in its [model card](https://huggingface.co/google/paligemma-3b-mix-448). ColPali inherits from this `gemma` license.
+
+## Contact
+
+- Manuel Faysse: [email protected]
+- Hugues Sibille: [email protected]
+- Tony Wu: [email protected]
+
+## Citation
+
+If you use any datasets or models from this organization in your research, please cite the original dataset as follows:
+
+```bibtex
+@misc{faysse2024colpaliefficientdocumentretrieval,
+  title={ColPali: Efficient Document Retrieval with Vision Language Models}, 
+  author={Manuel Faysse and Hugues Sibille and Tony Wu and Bilel Omrani and Gautier Viaud and Céline Hudelot and Pierre Colombo},
+  year={2024},
+  eprint={2407.01449},
+  archivePrefix={arXiv},
+  primaryClass={cs.IR},
+  url={https://arxiv.org/abs/2407.01449}, 
+}
+```

config.json

@@ -0,0 +1,53 @@
+{
+  "architectures": [
+    "ColPaliForRetrieval"
+  ],
+  "embedding_dim": 128,
+  "is_composition": false,
+  "model_type": "colpali",
+  "text_config": {
+    "hidden_size": 2048,
+    "intermediate_size": 16384,
+    "model_type": "gemma",
+    "num_attention_heads": 8,
+    "num_hidden_layers": 18,
+    "num_image_tokens": 1024,
+    "num_key_value_heads": 1,
+    "torch_dtype": "float32",
+    "vocab_size": 257216
+  },
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.48.0.dev0",
+  "vlm_config": {
+    "_name_or_path": "google/paligemma-3b-mix-448",
+    "_vocab_size": 257216,
+    "bos_token_id": 2,
+    "eos_token_id": 1,
+    "image_token_index": 257152,
+    "model_type": "paligemma",
+    "pad_token_id": 0,
+    "text_config": {
+      "hidden_size": 2048,
+      "intermediate_size": 16384,
+      "num_attention_heads": 8,
+      "num_hidden_layers": 18,
+      "num_image_tokens": 1024,
+      "num_key_value_heads": 1,
+      "torch_dtype": "float32",
+      "vocab_size": 257216
+    },
+    "torch_dtype": "bfloat16",
+    "vision_config": {
+      "hidden_size": 1152,
+      "image_size": 448,
+      "intermediate_size": 4304,
+      "num_attention_heads": 16,
+      "num_hidden_layers": 27,
+      "num_image_tokens": 1024,
+      "patch_size": 14,
+      "projection_dim": 2048,
+      "projector_hidden_act": "gelu_fast",
+      "vision_use_head": false
+    }
+  }
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4706678a55bb40df34f0e65939d7ee4cb6f41df1fac29a1fb9402d35a1d96a64
+size 4986816144

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d97e225d93f2bf10c89ea87495d6df1e1dd7004c07ad43d7b713aeeb36c982d
+size 862495464

model.safetensors.index.json

@@ -0,0 +1,612 @@
+{
+  "metadata": {
+    "total_size": 5849226976
+  },
+  "weight_map": {
+    "embedding_proj_layer.bias": "model-00002-of-00002.safetensors",
+    "embedding_proj_layer.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.15.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.16.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.17.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.language_model.model.norm.weight": "model-00002-of-00002.safetensors",
+    "vlm.multi_modal_projector.linear.bias": "model-00001-of-00002.safetensors",
+    "vlm.multi_modal_projector.linear.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.embeddings.patch_embedding.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.embeddings.patch_embedding.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.embeddings.position_embedding.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.21.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.22.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.23.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.24.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.25.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.26.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.layer_norm1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.layer_norm1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.layer_norm2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.layer_norm2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.mlp.fc1.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.mlp.fc1.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.mlp.fc2.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.mlp.fc2.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.out_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.out_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.encoder.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.post_layernorm.bias": "model-00001-of-00002.safetensors",
+    "vlm.vision_tower.vision_model.post_layernorm.weight": "model-00001-of-00002.safetensors"
+  }
+}

preprocessor_config.json

@@ -0,0 +1,25 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "SiglipImageProcessor",
+  "image_seq_length": 1024,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "processor_class": "ColPaliProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 448,
+    "width": 448
+  }
+}

special_tokens_map.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<image>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ff84f53c290d0348c4e206da6094ef781cf8c0e482fec8b268a996b32257cfd
+size 34600975