llm-semantic-router / mmbert-feedback-detector-lora

mmBERT Feedback Detector (LoRA Adapter)

A multilingual 4-class user feedback classifier built on jhu-clsp/mmBERT-base . This is the LoRA adapter version for parameter-efficient fine-tuning and inference.

Model Description

This model uses LoRA (Low-Rank Adaptation) for efficient fine-tuning, reducing trainable parameters to just 1.09% of the full model while maintaining high performance.

For the merged version (ready for direct inference), see llm-semantic-router/mmbert-feedback-detector-merged .

Labels

Performance

Per-Class Performance

LoRA Configuration

Multilingual Support

Thanks to mmBERT's multilingual pretraining (256k vocabulary, 100+ languages), this model achieves excellent cross-lingual transfer:

Usage

With PEFT

from peft import PeftModel, PeftConfig
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

# Load base model and LoRA adapter
base_model_name = "jhu-clsp/mmBERT-base"
adapter_name = "llm-semantic-router/mmbert-feedback-detector-lora"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_name)

# Load base model
base_model = AutoModelForSequenceClassification.from_pretrained(
    base_model_name,
    num_labels=4,
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

# Classify feedback
text = "Thanks, that's exactly what I needed!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    pred = probs.argmax().item()

labels = ["SAT", "NEED_CLARIFICATION", "WRONG_ANSWER", "WANT_DIFFERENT"]
print(f"Prediction: {labels[pred]} ({probs[0][pred]:.1%})")

Merge and Save

from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Load and merge
base_model = AutoModelForSequenceClassification.from_pretrained(
    "jhu-clsp/mmBERT-base",
    num_labels=4,
    trust_remote_code=True
)
model = PeftModel.from_pretrained(base_model, "llm-semantic-router/mmbert-feedback-detector-lora")
merged_model = model.merge_and_unload()

# Save merged model
merged_model.save_pretrained("./merged_model")
tokenizer = AutoTokenizer.from_pretrained("llm-semantic-router/mmbert-feedback-detector-lora")
tokenizer.save_pretrained("./merged_model")

Continue Fine-tuning

from peft import PeftModel, get_peft_model, LoraConfig
from transformers import AutoModelForSequenceClassification

# Load existing adapter
base_model = AutoModelForSequenceClassification.from_pretrained(
    "jhu-clsp/mmBERT-base",
    num_labels=4,
    trust_remote_code=True
)
model = PeftModel.from_pretrained(base_model, "llm-semantic-router/mmbert-feedback-detector-lora")

# Continue training with your data
# model.train()
# ... training code ...

Training Details

• Base Model : jhu-clsp/mmBERT-base
• Method : LoRA (Low-Rank Adaptation)
• LoRA Rank : 16
• LoRA Alpha : 32
• Learning Rate : 2e-5
• Batch Size : 32
• Epochs : 5
• Max Length : 512
• Dataset : llm-semantic-router/feedback-detector-dataset

Use Cases

• Conversational AI : Understand if users are satisfied with chatbot responses
• Customer Support : Route dissatisfied users to human agents
• Quality Monitoring : Track response quality across languages
• Feedback Analysis : Categorize user feedback automatically
• Continued Fine-tuning : Adapt to domain-specific feedback patterns

Advantages of LoRA

• Storage Efficient : Only ~13MB adapter vs ~1.2GB full model
• Fast Training : Train in minutes on consumer GPUs
• Composable : Stack with other adapters
• Base Model Updates : Benefit from base model improvements

Related Models

• llm-semantic-router/mmbert-feedback-detector-merged - Merged version
• llm-semantic-router/mmbert-intent-classifier-lora - Intent classification
• llm-semantic-router/mmbert-fact-check-lora - Fact checking
• llm-semantic-router/mmbert-jailbreak-detector-lora - Security

Citation

@misc{mmbert-feedback-detector-lora,
  title={mmBERT Feedback Detector LoRA Adapter},
  author={vLLM Semantic Router Team},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/llm-semantic-router/mmbert-feedback-detector-lora}
}

License

Apache 2.0