merve / multilabel-v1-replica

Train-Test Set: "intent-multilabel-v1-2.zip"

Model: "dbmdz/bert-base-turkish-cased"

Tokenizer Params

max_length=128
padding="max_length"
truncation=True

Training Params

evaluation_strategy = "epoch"
save_strategy = "epoch"
per_device_train_batch_size = 16
per_device_eval_batch_size = 16
num_train_epochs = 4
load_best_model_at_end = True

Train-Val Splitting Configuration

train_test_split(df_train,
                 test_size=0.1,
                 random_state=1111)

Training Log

Epoch	Training Loss	Validation Loss
1	    No log	        0.150276
2	    0.195100	    0.132906
3	    0.107700	    0.128633
4	    0.107700	    0.127795

Threshold Optimization

• Best Threshold: 0.1
• F1 @ Threshold: 0.734

Eval Results

              precision    recall  f1-score   support

    Alakasiz       0.90      0.87      0.89       734
     Barinma       0.85      0.80      0.83       207
  Elektronik       0.73      0.78      0.75       130
       Giysi       0.83      0.66      0.73        94
    Kurtarma       0.86      0.79      0.82       362
    Lojistik       0.73      0.51      0.60       112
      Saglik       0.74      0.74      0.74       108
          Su       0.64      0.60      0.62        78
       Yagma       0.68      0.55      0.61        31
       Yemek       0.80      0.83      0.81       117

   micro avg       0.84      0.79      0.81      1973
   macro avg       0.78      0.71      0.74      1973
weighted avg       0.84      0.79      0.81      1973
 samples avg       0.84      0.82      0.82      1973