yasserrmd / pediatrics-gemma-300m-emb

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m . It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yasserrmd/pediatrics-gemma-300m-emb")
# Run inference
queries = [
    "How can interdisciplinary collaboration and clearly defined strategies help reduce damage and suffering for children and adolescents with mental disorders and their families?\n",
]
documents = [
    'Interdisciplinary collaboration, involving caregivers, mental health managers, and professionals, can play a crucial role in reducing damage and suffering for children and adolescents with mental disorders and their families. By developing a shared vision and practice, professionals can work together to establish dynamic strategies that address the complex demands of these individuals. This collaboration requires critical observation of services, conduct, and strategies to ensure that families and mental health services are brought closer together. By bridging the gap between demands and care, interdisciplinary collaboration can help improve the well-being and outcomes for children, adolescents, and their families.',
    'Pneumothorax in children can have various causes, including lung cysts. In cases where there are signs of pneumothorax without a lung lesion to account for the condition, cysts of the lung should be suspected. The presence of cysts can be confirmed through radiographic evidence, which can also help determine the location and characteristics of the cysts.',
    'The infant in the NICU presented with a degraded general status, intubation, and mechanical ventilation. They also had unilaterally diminished breath sounds, hypoxemia, oliguria, tachycardia, hypotension, abdominal distention, and fever. Additionally, they exhibited hepatosplenomegaly, oliguria progressing to anuria, thrombocytopenia, and hyperchromic hematuria. ',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.7673, -0.0352,  0.0221]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 20,000 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 8 tokens mean: 20.38 tokens max: 64 tokens	min: 17 tokens mean: 87.92 tokens max: 258 tokens

• Samples:

  sentence_0	sentence_1
  What is the role of routine health check-ups in detecting and diagnosing metabolic syndrome and NAFLD in obese children?	Routine health check-ups are important in detecting and diagnosing metabolic syndrome and NAFLD in obese children. However, there is a lack of routine health check-up data specifically for these complications in obese children. To address this need, pediatric health promotion centers and pediatric obesity clinics have been developed. The aim of these centers is to provide routine health check-ups and obesity-oriented check-ups to detect and diagnose metabolic syndrome and NAFLD in children.
  How does the implementation of family-centered rounds (FCR) impact medical education?	The implementation of family-centered rounds (FCR) has raised concerns about its potential impact on medical education. Some evidence suggests that FCR may lead to decreased "didactic" teaching, increased discomfort in asking specific management questions, and limited time to discuss management options for residents and students (8, 9, 10, 11). However, the literature on the association between FCR and teaching has mainly focused on learners' perceptions, and there is a lack of objective data to address the relationship between FCR and medical knowledge acquisition.
  What are some common clinical symptoms of neonatal septicaemia?	Some common clinical symptoms of neonatal septicaemia include fever, poor feeding, excessive cry, difficulty in breathing, yellowish skin discoloration, skin rashes, jitteriness, and irritability.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size : 6
• per_device_eval_batch_size : 6
• num_train_epochs : 1
• multi_dataset_batch_sampler : round_robin

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.56.2
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}