davanstrien / iconclass-retriever-bge-ft

Iconclass retriever (bge-base-en-v1.5 fine-tuned)

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the iconclass-retriever-pairs dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • iconclass-retriever-pairs
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("davanstrien/iconclass-retriever-bge-ft")
# Run inference
queries = [
    'An engraving depicts a dramatic nocturnal scene where a man in period clothing stands on a burning ladder, reaching up to assist a young boy who is climbing down from a high window. The ladder is engulfed in flames at its base, creating a sense of peril, while the boy looks down with a calm expression. In the background, a cityscape is visible under a night sky featuring a radiant sun or moon.',
]
documents = [
    'burning',
    'glowing (heat)',
    'radiating (heat)',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5269, 0.3512, 0.3070]])

Evaluation

Metrics

Information Retrieval

• Dataset: iconclass
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

iconclass-retriever-pairs

• Dataset: iconclass-retriever-pairs
• Size: 21,541 training samples
• Columns: anchor , positive , negative_1 , negative_2 , negative_3 , and negative_4
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4
  type	string	string	string	string	string	string
  modality	text	text	text	text	text	text
  details	min: 58 tokens mean: 81.44 tokens max: 116 tokens	min: 3 tokens mean: 6.9 tokens max: 26 tokens	min: 3 tokens mean: 7.86 tokens max: 27 tokens	min: 3 tokens mean: 8.91 tokens max: 30 tokens	min: 3 tokens mean: 8.61 tokens max: 28 tokens	min: 3 tokens mean: 7.29 tokens max: 24 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4
  A dimly lit interior scene depicts a group of horses and a donkey within a rustic stable or ruin. A white horse stands prominently in the center, flanked by a darker horse to its left and another dark horse to its right. In the foreground, a donkey stands near a pile of discarded tack and blankets on the ground, while a small arched window on the left provides a source of light.	barn	farm (building) - AA - civic architecture: inside	saints	imps, familiars of witches and sorcerers	horses (circus performance)
  A vibrant bouquet of various flowers, including roses, tulips, and a hanging crown imperial, is arranged in a dark vase on a wooden ledge. The composition is animated by several butterflies fluttering among the blooms and a small bird's nest with eggs resting on the ledge to the left. A single red tulip lies fallen on the ledge in the foreground, adding a sense of transience to the scene.	flowers	artificial plants and flowers	fabulous plants and herbs	bunch of flowers ~ ornament - AA - stylized	flowers ~ ornament
  A panoramic landscape painting depicts a wide river or lake reflecting the sky, flanked by dense clusters of trees and foliage. In the foreground on the left, a group of travelers walks along a dirt path, while a solitary figure stands nearby. In the background, a town with a prominent church spire and other buildings is visible across the water.	water course	canal	sea (seascape)	landscapes with waters, waterscapes, seascapes (in the temperate zone) - HH - ideal landscapes	heaven represented as paradisaical landscape

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Evaluation Dataset

iconclass-retriever-pairs

• Dataset: iconclass-retriever-pairs
• Size: 200 evaluation samples
• Columns: anchor , positive , negative_1 , negative_2 , negative_3 , and negative_4
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3	negative_4
  type	string	string	string	string	string	string
  modality	text	text	text	text	text	text
  details	min: 50 tokens mean: 79.51 tokens max: 111 tokens	min: 3 tokens mean: 8.2 tokens max: 33 tokens	min: 3 tokens mean: 8.24 tokens max: 50 tokens	min: 3 tokens mean: 8.41 tokens max: 41 tokens	min: 3 tokens mean: 7.78 tokens max: 25 tokens	min: 3 tokens mean: 7.59 tokens max: 20 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3	negative_4
  A muscular, nude male figure lies supine on a rocky outcrop, his body contorted in a dramatic pose with his head thrown back and his right arm raised toward the sky. A second figure, appearing to be a woman with flowing hair, stands beside him, leaning over and reaching out to touch or support his raised hand. The scene is set against a backdrop of heavy, swirling clouds and a distant landscape, creating a tense and theatrical atmosphere.	David thanks the Lord after he has slain the giant Goliath	David speaks with Goliath	David slings a stone at Goliath's forehead	David beheads Goliath with a sword	lying figure
  A seated female figure, representing the element of water, is depicted in profile facing left. She holds a long, slender vessel from which she pours a stream of water, while a large basket filled with reeds or rushes rests beside her on a rock.	island	dale, valley	coast	forest, wood	plain
  The Virgin Mary, wearing a blue mantle and red dress, sits enthroned holding the Christ Child on her lap. To the left, a Dominican nun and a Dominican friar kneel in prayer, looking up at the holy figures. Above them, a heavenly host of angels and cherubs gathers in a burst of golden light, while two putti at the base of the throne arrange a floral garland.	flowers	artificial plants and flowers	fabulous plants and herbs	standing figure	stand-in

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size : 64
• learning_rate : 2e-05
• warmup_steps : 0.1
• weight_decay : 0.01
• bf16 : True
• per_device_eval_batch_size : 128
• push_to_hub : True
• hub_model_id : davanstrien/iconclass-retriever-bge-ft
• load_best_model_at_end : True
• seed : 12
• batch_sampler : no_duplicates

All Hyperparameters

Training Logs

Training Time

• Training : 8.1 minutes
• Evaluation : 18.6 seconds
• Total : 8.4 minutes

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.5.1
• Transformers: 5.10.2
• PyTorch: 2.12.0+cu130
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}