llm-semantic-router / mlcommons-safety-classifier-level2-hazard

MLCommons AI Safety Classifier - Level 2 (9-Class Hazard)

A LoRA-finetuned multilingual BERT model for 9-class hazard category classification, following the MLCommons AI Safety Hazard Taxonomy.

Model Description

This is Level 2 of a hierarchical safety classification system:

• Level 1 : Binary classification (safe vs unsafe)
• Level 2 (this model) : 9-class hazard category classification

The model uses mmBERT (Multilingual ModernBERT) as the base, supporting 1800+ languages.

Training Results

Hazard Categories (MLCommons AI Safety Taxonomy)

Training Data

• Total samples : ~20,000 (balanced across categories)
• Sources :
  • AEGIS AI Content Safety Dataset 2.0 (~18,000 samples)
  • MLCommons AI Safety Synth (12,000 synthesized samples for weak categories)

Synthesized Data Distribution

The synthetic dataset targets previously underrepresented categories:

• S2 (Non-Violent Crimes): 2,000 samples
• S6 (Self-Harm): 2,000 samples
• S7 (Hate): 2,000 samples
• S9 (Privacy): 2,000 samples
• S11 (Elections): 2,000 samples
• S13 (Misinformation): 2,000 samples

Model Architecture & Training

Base Model

• Model : jhu-clsp/mmBERT-base
• Architecture : ModernBERT (314M parameters)

LoRA Configuration

Training Hyperparameters

Hardware & Environment

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

# Load base model and tokenizer
base_model = "jhu-clsp/mmBERT-base"
tokenizer = AutoTokenizer.from_pretrained("llm-semantic-router/mlcommons-safety-classifier-level2-hazard")
model = AutoModelForSequenceClassification.from_pretrained(base_model, num_labels=9)
model = PeftModel.from_pretrained(model, "llm-semantic-router/mlcommons-safety-classifier-level2-hazard")

# Classify
text = "How to hack into someone's email account"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
outputs = model(**inputs)
prediction = outputs.logits.argmax(-1).item()

# Label mapping
labels = [
    "S1_violent_crimes", "S2_nonviolent_crimes", "S3_sex_crimes",
    "S5_weapons_cbrne", "S6_self_harm", "S7_hate",
    "S8_specialized_advice", "S9_privacy", "S13_misinformation"
]
print(f"Hazard Category: {labels[prediction]}")

Label Mapping

{
  "S1_violent_crimes": 0,
  "S2_nonviolent_crimes": 1,
  "S3_sex_crimes": 2,
  "S5_weapons_cbrne": 3,
  "S6_self_harm": 4,
  "S7_hate": 5,
  "S8_specialized_advice": 6,
  "S9_privacy": 7,
  "S13_misinformation": 8
}

Hierarchical Usage (Recommended)

For production use, combine Level 1 and Level 2:

# Step 1: Binary classification (Level 1)
level1_pred = level1_model(inputs)
if level1_pred == "unsafe":
    # Step 2: Hazard classification (Level 2)
    hazard_category = level2_model(inputs)

Intended Use

This model is designed for:

• Detailed hazard categorization of unsafe content
• Content moderation with specific policy enforcement
• Safety analytics and reporting
• Research on content safety classification

Limitations

• Optimized for English but supports 1800+ languages via mmBERT
• Should be used after Level 1 filtering for efficiency
• Some categories may have regional/cultural variations
• May require domain-specific fine-tuning for specialized applications

Citation

@misc{mlcommons-safety-classifier,
  title={MLCommons AI Safety Classifier},
  author={LLM Semantic Router Team},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/llm-semantic-router/mlcommons-safety-classifier-level2-hazard}
}

License

Apache 2.0