LLukas22 / all-mpnet-base-v2-embedding-all

all-mpnet-base-v2-embedding-all

This model is a fine-tuned version of all-mpnet-base-v2 on the following datasets: squad , newsqa , LLukas22/cqadupstack , LLukas22/fiqa , LLukas22/scidocs , deepset/germanquad , LLukas22/nq .

Usage (Sentence-Transformers)

Using this model becomes easy when you have sentence-transformers installed:

pip install -U sentence-transformers

Then you can use the model like this:

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('LLukas22/all-mpnet-base-v2-embedding-all')
embeddings = model.encode(sentences)
print(embeddings)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1E+00
• per device batch size: 60
• effective batch size: 180
• seed: 42
• optimizer: AdamW with betas (0.9,0.999) and eps 1E-08
• weight decay: 2E-02
• D-Adaptation: True
• Warmup: True
• number of epochs: 15
• mixed_precision_training: bf16

Training results

Evaluation results

Framework versions

• Transformers: 4.25.1
• PyTorch: 2.0.0.dev20230210+cu118
• PyTorch Lightning: 1.8.6
• Datasets: 2.7.1
• Tokenizers: 0.13.1
• Sentence Transformers: 2.2.2

Additional Information

This model was trained as part of my Master's Thesis 'Evaluation of transformer based language models for use in service information systems' . The source code is available on Github .