Ana2012 / bertimbau-buscador

SentenceTransformer based on neuralmind/bert-base-portuguese-cased

This is a sentence-transformers model finetuned from neuralmind/bert-base-portuguese-cased . It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: neuralmind/bert-base-portuguese-cased
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Salgados',
    'Assado Integral de Brócolis',
    'STROGONOFF DE FRANGO',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6608, 0.4083],
#         [0.6608, 1.0000, 0.4988],
#         [0.4083, 0.4988, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 17 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 17 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 4 tokens mean: 6.41 tokens max: 9 tokens	min: 5 tokens mean: 11.29 tokens max: 24 tokens

• Samples:

  sentence_0	sentence_1
  Pastel de pizza	Pastel de Pizza
  quero massa carbonara	Massa - Carbonara
  Açai com granola	Açaí 400ml com Frutas + Granola + Leite em Pó

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size : 16
• per_device_eval_batch_size : 16
• num_train_epochs : 2
• multi_dataset_batch_sampler : round_robin

All Hyperparameters

Click to expand

• do_predict : False
• eval_strategy : no
• prediction_loss_only : True
• per_device_train_batch_size : 16
• per_device_eval_batch_size : 16
• gradient_accumulation_steps : 1
• eval_accumulation_steps : None
• torch_empty_cache_steps : None
• learning_rate : 5e-05
• weight_decay : 0.0
• adam_beta1 : 0.9
• adam_beta2 : 0.999
• adam_epsilon : 1e-08
• max_grad_norm : 1
• num_train_epochs : 2
• max_steps : -1
• lr_scheduler_type : linear
• lr_scheduler_kwargs : None
• warmup_ratio : None
• warmup_steps : 0
• log_level : passive
• log_level_replica : warning
• log_on_each_node : True
• logging_nan_inf_filter : True
• enable_jit_checkpoint : False
• save_on_each_node : False
• save_only_model : False
• restore_callback_states_from_checkpoint : False
• use_cpu : False
• seed : 42
• data_seed : None
• bf16 : False
• fp16 : False
• bf16_full_eval : False
• fp16_full_eval : False
• tf32 : None
• local_rank : -1
• ddp_backend : None
• debug : []
• dataloader_drop_last : False
• dataloader_num_workers : 0
• dataloader_prefetch_factor : None
• disable_tqdm : False
• remove_unused_columns : True
• label_names : None
• load_best_model_at_end : False
• ignore_data_skip : False
• fsdp : []
• fsdp_config : {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config : {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config : None
• deepspeed : None
• label_smoothing_factor : 0.0
• optim : adamw_torch_fused
• optim_args : None
• group_by_length : False
• length_column_name : length
• project : huggingface
• trackio_space_id : trackio
• ddp_find_unused_parameters : None
• ddp_bucket_cap_mb : None
• ddp_broadcast_buffers : False
• dataloader_pin_memory : True
• dataloader_persistent_workers : False
• skip_memory_metrics : True
• push_to_hub : False
• resume_from_checkpoint : None
• hub_model_id : None
• hub_strategy : every_save
• hub_private_repo : None
• hub_always_push : False
• hub_revision : None
• gradient_checkpointing : False
• gradient_checkpointing_kwargs : None
• include_for_metrics : []
• eval_do_concat_batches : True
• auto_find_batch_size : False
• full_determinism : False
• ddp_timeout : 1800
• torch_compile : False
• torch_compile_backend : None
• torch_compile_mode : None
• include_num_input_tokens_seen : no
• neftune_noise_alpha : None
• optim_target_modules : None
• batch_eval_metrics : False
• eval_on_start : False
• use_liger_kernel : False
• liger_kernel_config : None
• eval_use_gather_object : False
• average_tokens_across_devices : True
• use_cache : False
• prompts : None
• batch_sampler : batch_sampler
• multi_dataset_batch_sampler : round_robin
• router_mapping : {}
• learning_rate_mapping : {}

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.3.0
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}