dalek-ai / minilm-next-v1

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 . It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'I need information on private equity firms in New York, focusing on the technology industry. Also, can you get details on a couple of private equity deals involving companies named "TechSoft" and "Digittal", deals are sized 500 and 700 million in the USA. Lastly, I need more information on a private equity firm named "CapitalGrow".',
    'Retrieve information about a private equity deal',
    'Analyze stock market data for a given company',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6397, 0.3693],
#         [0.6397, 1.0000, 0.3183],
#         [0.3693, 0.3183, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 8,386 training samples
• Columns: sentence_0 , sentence_1 , and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 12 tokens mean: 101.35 tokens max: 128 tokens	min: 6 tokens mean: 22.63 tokens max: 116 tokens	min: 5 tokens mean: 18.8 tokens max: 116 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  I'm responsible for managing the inventory of the 'NextGen Biosciences' research facility. We need to track the stock of essential items like PCR machines, reagents, and consumables to ensure smooth operation of our ongoing experiments. We have a JSON file that lists our current inventory status. The actions we want to perform include: updating the system when an item is used or restocked, receiving alerts when an item's stock falls below a certain threshold, and generating various reports for stock, consumption, and purchase history. Our JSON inventory list is located at "/path/to/life_sciences_inventory.json". For instance, we recently purchased 5 additional ultra-centrifuges, each with a unique item ID. Moreover, we regularly consume gloves and need to reduce our stock count by 100 units. We want to: 1. Update the inventory for the ultra-centrifuges purchase. 2. Update the inventory for the consumption of gloves. 3. Get alerts for any stock below 10 units. 4. Generate a stock summ...	Generates alerts for items in the inventory that fall below a specified stock threshold.	Update the inventory of equipment on a boat.
  Role definition: Inquirer: A user who raises an inquiry. Response assistant: Communicates with the inquirer and provides answers and solutions. Historical dialog data is as follows: Inquirer: We recently found skeletal remains at a crime scene, and I need your help to determine certain details. I need to know the time since death using the ambient temperature, which was recorded as 10 Celsius, and to determine the cause of death based on the skeletal remains. Additionally, I seek to identify the remains based on the skeletal elements found, which include a skull and femur. Response assistant: Could you please provide more specific details about the corpse involved for estimating time since death and determining the cause of death? Meanwhile, I will proceed with identifying the human remains based on the skull and femur you mentioned. Inquirer: The corpse is mostly decomposed. It is believed to have been at the site for a prolonged period. Please continue your answer given the his...	Identify human remains based on forensic anthropology	Generates a report on the response times for customer complaints.
  You are Isabella Johansson, and you live in 32286. You want to return the skateboard, garden hose, backpack, keyboard, bed, and also cancel the hose you just ordered (if cancelling one item is not possible, forget about it, you just want to cancel the hose and nothing else). You want to know how much you can get in total as refund. You are extremely brief but patient.	Retrieves detailed information about a specific order by its ID in the Order Management System.	Cancels a transaction that has been initiated but not yet confirmed.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size : 64
• per_device_eval_batch_size : 64
• num_train_epochs : 2
• multi_dataset_batch_sampler : round_robin

All Hyperparameters

Framework Versions

• Python: 3.9.6
• Sentence Transformers: 5.1.2
• Transformers: 4.57.6
• PyTorch: 2.8.0
• Accelerate: 1.10.1
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}