AventIQ-AI / distilBERT_Sentiment_Analysis

DistilBERT Quantized Model for IMDB Sentiment Analysis

This repository contains a quantized DistilBERT model fine-tuned for binary sentiment classification on IMDB movie reviews. Optimized for production deployment, the model achieves high accuracy while maintaining efficiency.

Model Details

• Model Architecture: DistilBERT Base Uncased
• Task: Binary Sentiment Analysis (Positive/Negative)
• Dataset: IMDB Movie Reviews (50K samples)
• Quantization: Dynamic Quantization (INT8)
• Framework: Hugging Face Transformers + PyTorch

Usage

Installation

pip install transformers torch scikit-learn pandas

Loading the Model

from transformers import DistilBertForSequenceClassification, DistilBertTokenizer
import torch

# Load quantized model
model_path = "./quantized_sentiment_model.pth"
model = DistilBertForSequenceClassification.from_pretrained("./sentiment_model")
model.load_state_dict(torch.load(model_path))
model.eval()

# Load tokenizer
tokenizer = DistilBertTokenizer.from_pretrained("./sentiment_model")

def predict_sentiment(text):
    inputs = tokenizer(text, return_tensors="pt", 
                      padding=True, truncation=True, 
                      max_length=128)
    
    with torch.no_grad():
        outputs = model(**inputs)
    
    prediction = torch.argmax(outputs.logits).item()
    return "Positive" if prediction == 1 else "Negative"

# Example usage
review = "This movie blew me away with its stunning visuals and gripping storyline."
print(predict_sentiment(review))  # Output: Positive

📊 Performance Metrics

🏋️ Training Details

Dataset

• 50,000 IMDB movie reviews
• Balanced binary classes (50% positive, 50% negative)

Hyperparameters

• Epochs: 5
• Batch Size: 24 (Effective 48 with accumulation)
• Learning Rate: 8e-6
• Warmup Ratio: 10%
• Weight Decay: 0.005
• Optimizer: AdamW with Cosine LR Schedule

Quantization

Applied dynamic post-training quantization:

quantized_model = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

📁 Repository Structure

.
├── sentiment_model/                # Full-precision model files
│   ├── config.json
│   ├── pytorch_model.bin
│   └── tokenizer files...
├── quantized_sentiment_model.pth  # Quantized weights
├── imdb_train.csv                 # Sample training data
├── train.py                       # Training script
└── inference.py                   # Usage examples

⚠️ Limitations

• Accuracy may drop on reviews with:
  • Sarcasm or nuanced language
  • Domain-specific terminology (non-movie content)
• Maximum sequence length: 128 tokens
• English language only