rasyosef / splade-mini

Metrics

Sparse Information Retrieval

• Evaluated with SparseInformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 250,000 training samples
• Columns: query , positive , negative_1 , negative_2 , negative_3 , negative_4 , and label
• Approximate statistics based on the first 1000 samples:

  	query	positive	negative_1	negative_2	negative_3	negative_4	label
  type	string	string	string	string	string	string	list
  details	min: 4 tokens mean: 8.87 tokens max: 43 tokens	min: 24 tokens mean: 81.23 tokens max: 259 tokens	min: 20 tokens mean: 79.21 tokens max: 197 tokens	min: 20 tokens mean: 77.89 tokens max: 207 tokens	min: 18 tokens mean: 76.38 tokens max: 271 tokens	min: 18 tokens mean: 75.46 tokens max: 214 tokens	size: 4 elements

• Samples:

  query	positive	negative_1	negative_2	negative_3	negative_4	label
  heart specialists in ridgeland ms	Dr. George Reynolds Jr, MD is a cardiology specialist in Ridgeland, MS and has been practicing for 35 years. He graduated from Vanderbilt University School Of Medicine in 1977 and specializes in cardiology and internal medicine.	Dr. James Kramer is a Internist in Ridgeland, MS. Find Dr. Kramer's phone number, address and more.	Dr. James Kramer is an internist in Ridgeland, Mississippi. He received his medical degree from Loma Linda University School of Medicine and has been in practice for more than 20 years. Dr. James Kramer's Details	Chronic Pulmonary Heart Diseases (incl. Pulmonary Hypertension) Coarctation of the Aorta; Congenital Aortic Valve Disorders; Congenital Heart Defects; Congenital Heart Disease; Congestive Heart Failure; Coronary Artery Disease (CAD) Endocarditis; Heart Attack (Acute Myocardial Infarction) Heart Disease; Heart Murmur; Heart Palpitations; Hyperlipidemia; Hypertension	A growing shortage of primary care doctors means you might have to look harder for ongoing care. How to Read an OTC Medication Label Purvi Parikh, M.D.	Feb. 12, 2018
  does baytril otic require a prescription	Baytril Otic Ear Drops-Enrofloxacin/Silver Sulfadiazine-Prices & Information. A prescription is required for this item. A prescription is required for this item. Brand medication is not available at this time.	RX required for this item. Click here for our full Prescription Policy and Form. Baytril Otic (enrofloxacin/silver sulfadiazine) Emulsion from Bayer is the first fluoroquinolone approved by the Food and Drug Administration for the topical treatment of canine otitis externa.	Product Details. Baytril Otic is a highly effective treatment prescribed by many veterinarians when your pet has an ear infection caused by susceptible bacteria or fungus. Baytril Otic is: a liquid emulsion that is used topically directly in the ear or on the skin in order to treat susceptible bacterial and yeast infections.	Baytril for dogs is an antibiotic often prescribed for bacterial infections, particularly those involving the ears. Ear infections are rare in many animals, but quite common in dogs. This is particularly true for dogs with long droopy ears, where it will stay very warm and moist.	Administer 5-10 Baytril ear drops per treatment in dogs 35 lbs or less and 10-15 drops per treatment in dogs more than 35 lbs.	[1.0, 3.640146493911743, 6.450072288513184, 11.96937084197998]
  what is on a gyro	Report Abuse. Gyros or gyro (giros) (pronounced /ˈjɪəroʊ/ or /ˈdʒaɪroʊ/, Greek: γύρος turn) is a Greek dish consisting of meat (typically lamb and/or beef), tomato, onion, and tzatziki sauce, and is served with pita bread. Chicken and pork meat can be used too.	A gyroscope (from Ancient Greek γῦρος gûros, circle and σκοπέω skopéō, to look) is a spinning wheel or disc in which the axis of rotation is free to assume any orientation by itself. When rotating, the orientation of this axis is unaffected by tilting or rotation of the mounting, according to the conservation of angular momentum.	Diagram of a gyro wheel. Reaction arrows about the output axis (blue) correspond to forces applied about the input axis (green), and vice versa. A gyroscope is a wheel mounted in two or three gimbals, which are a pivoted supports that allow the rotation of the wheel about a single axis.	A fair number of our users are unsure of how to pronounce gyro. This isn't surprising, since there are two different gyros and they have two different pronunciations. The earlier gyro is the one that is a shortened form of gyrocompass or gyroscope, and it has a pronunciation that conforms to one's expectations: /JEYE-roh/.	Vibration Gyro Sensors. Vibration gyro sensors sense angular velocity from the Coriolis force applied to a vibrating element. For this reason, the accuracy with which angular velocity is measured differs significantly depending on element material and structural differences.	[2.1750364303588867, 2.634796142578125, 4.30520486831665, 6.382436752319336]

• Loss: SpladeLoss with these parameters:

  {
      "loss": "SparseMarginMSELoss",
      "document_regularizer_weight": 0.3,
      "query_regularizer_weight": 0.5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy : epoch
• per_device_train_batch_size : 48
• per_device_eval_batch_size : 48
• learning_rate : 8e-05
• num_train_epochs : 6
• lr_scheduler_type : cosine
• warmup_ratio : 0.025
• fp16 : True
• load_best_model_at_end : True
• optim : adamw_torch_fused

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.0.0
• Transformers: 4.53.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.8.1
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

SpladeLoss

@misc{formal2022distillationhardnegativesampling,
      title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
      author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
      year={2022},
      eprint={2205.04733},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2205.04733},
}

SparseMarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}

FlopsLoss

@article{paria2020minimizing,
    title={Minimizing flops to learn efficient sparse representations},
    author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
    journal={arXiv preprint arXiv:2004.05665},
    year={2020}
}