Omartificial-Intelligence-Space / Qwen3-VL-Embedding-2B-Arabic-VDR

Qwen3-VL-Embedding-2B finetuned on Arabic-culture visual document retrieval (Pearl-vdr-ar), 3 epochs total

This is a sentence-transformers model trained on the pearl-vdr-ar-train-preprocessed dataset. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Maximum Sequence Length: 262144 tokens
• Output Dimensionality: 2048 dimensions
• Similarity Function: Cosine Similarity
• Supported Modalities: Text, Image, Video, Message
• Training Dataset:
  • pearl-vdr-ar-train-preprocessed
• Language: ar
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'image': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'video': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'message': {'method': 'forward', 'method_output_name': 'last_hidden_state', 'format': 'structured'}}, 'module_output_name': 'token_embeddings', 'processing_kwargs': {'chat_template': {'add_generation_prompt': True}}, 'unpad_inputs': False, 'architecture': 'Qwen3VLModel'})
  (1): Pooling({'embedding_dimension': 2048, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'ما هو الغطاء الرأس الذي يعكس الهوية والمكانة الاجتماعية كما يظهر في الصورة؟',
]
documents = [
    'Clothes',
    'Music',
    'Architecture',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 2048] [3, 2048]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.2472, 0.1607, 0.1453]])

Evaluation

Metrics

Information Retrieval

• Datasets: pearl-ar-dev-hard , pearl-ar-dev-hard-final and pearl-ar-test-hard-final
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

pearl-vdr-ar-train-preprocessed

• Dataset: pearl-vdr-ar-train-preprocessed at 494822e
• Size: 48,002 training samples
• Columns: query , image , and negative_0
• Approximate statistics based on the first 1000 samples:

  	query	image	negative_0
  type	string	image	image
  details	min: 31 tokens mean: 51.45 tokens max: 90 tokens	min: 53x96 px mean: 639x540 px max: 800x798 px	min: 101x100 px mean: 630x545 px max: 800x787 px

• Samples:

  query	image	negative_0
  ما هي التحديات التي تواجه الحرف التقليدية كما يظهر في الصورة، وما هي الحلول الممكنة لمواجهة هذه التحديات؟
  إذا شاركت في ورشة عمل لتعلم كيفية صنع الآلة التي يظهر في الصورة، ما هي الخطوات التي ستحتاج إلى اتباعها لصنعها بشكل صحيح؟
  كيف يختلف العزف على الآلة التي يظهر في الصورة عن العزف على الآلات الوترية الأخرى في المنطقة، وما هي الخصائص الفريدة لهذه الآلة؟

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Dataset

pearl-vdr-ar-train-preprocessed

• Dataset: pearl-vdr-ar-train-preprocessed at 494822e
• Size: 200 evaluation samples
• Columns: query , category , country , image , negative_0 , negative_1 , negative_2 , and negative_3
• Approximate statistics based on the first 200 samples:

  	query	category	country	image	negative_0	negative_1	negative_2	negative_3
  type	string	string	string	image	image	image	image	image
  details	min: 33 tokens mean: 52.18 tokens max: 82 tokens	min: 21 tokens mean: 21.8 tokens max: 24 tokens	min: 21 tokens mean: 22.21 tokens max: 24 tokens	min: 200x135 px mean: 617x552 px max: 800x786 px	min: 168x182 px mean: 606x545 px max: 800x788 px	min: 141x139 px mean: 671x539 px max: 800x791 px	min: 177x175 px mean: 649x555 px max: 800x788 px	min: 200x150 px mean: 664x519 px max: 800x790 px

• Samples:

  query	category	country	image	negative_0	negative_1	negative_2	negative_3
  ما هو الغطاء الرأس الذي يعكس الهوية والمكانة الاجتماعية كما يظهر في الصورة؟	Clothes	Syria
  كيف تساهم المجوهرات في إبراز شخصية الفنانة كما يظهر في الصورة؟	Music	Lebanon
  ما هي الاختلافات بين الزخارف والكتابات التي تظهر في الصورة وبين تلك الموجودة في المعالم التاريخية الأخرى في نفس البلد، وما هي العناصر الفريدة التي تميز هذه الزخارف والكتابات؟	Architecture	Tunisia

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size : 64
• num_train_epochs : 2
• learning_rate : 1e-05
• warmup_steps : 0.03
• bf16 : True
• per_device_eval_batch_size : 64
• batch_sampler : no_duplicates

All Hyperparameters

Training Logs

Training Time

• Training : 8.7 hours
• Evaluation : 1.1 hours
• Total : 9.8 hours

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.4.1
• Transformers: 5.5.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.2.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}