dejanseo / ecommerce-query-volume-classifier

eCommerce Query Volume Classifier

A fine-tuned DeBERTa v3 base model that predicts the search volume class of ecommerce product queries. Trained on 39.6 million unique queries from the Amazon Shopping Queries dataset spanning 395.5 million search sessions.

Blog post: Is Query Length a Reliable Predictor of Search Volume?

Model Description

This model classifies ecommerce search queries into five volume tiers based on their expected search popularity:

The model learns semantic signals — brand recognition, category head terms, specificity markers — rather than superficial features like query length. Simple character/word-count heuristics achieve only ~25% accuracy on this task (barely above the 20% random baseline), while this model achieves 72.1% accuracy .

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "dejanseo/ecommerce-query-volume-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

labels = ["very_high", "high", "medium", "low", "very_low"]

queries = [
    "airpods",
    "wireless mouse",
    "organic flurb capsules",
    "replacement gasket for instant pot duo 8 quart",
]

inputs = tokenizer(queries, return_tensors="pt", padding=True, truncation=True, max_length=32)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=-1)
    preds = torch.argmax(probs, dim=-1)

for query, pred, prob in zip(queries, preds, probs):
    label = labels[pred.item()]
    confidence = prob[pred.item()].item() * 100
    print(f"{query:50s} → {label:>10s}  ({confidence:.1f}%)")

Performance

Evaluation (25K balanced sample, 5K per class)

Per-Class F1 Scores (best validation checkpoint)

The model performs best on the extremes (very high and very low volume) and struggles most with the low class, which sits in an ambiguous zone between medium and very_low .

Training Details

Hyperparameters

Sampling Strategy

Balanced sampling per epoch with different random seeds:

Total per epoch: 324,000 train / 36,000 validation

Training Curves

Validation Curves

Hardware

• GPU: NVIDIA GeForce RTX 4090 (24 GB)
• RAM: 128 GB
• OS: Windows 11
• Training time: ~2 hours 16 minutes
• Framework: PyTorch + Transformers 4.57.1

Dataset

Amazon Shopping Queries (AmazonQAC) — 395.5 million sessions, 39.6 million unique queries. Volume classes derived from raw occurrence counts across sessions.

What the Model Learns

The model captures semantic patterns rather than surface-level features like query length:

• Brand recognition: "airpods" → very high, regardless of character count
• Category head terms: "laptop", "headphones", "dog food" → recognized as high-volume entry points
• Specificity markers: Size specs, compatibility constraints, and material callouts signal niche demand
• Nonsense detection: Gibberish queries like "blorf" and "wireless blorf adapter" are correctly classified as very low volume, confirming the model isn't just counting characters

Limitations

• Trained exclusively on Amazon product search queries — may not generalize well to Google web search, informational queries, or non-English markets
• The low volume class is the weakest (F1 ≈ 0.39), reflecting genuine ambiguity in the boundary between medium and very low volume queries
• Volume thresholds are based on the Amazon QAC dataset's session counts, which may not map directly to other volume scales (e.g. Google Keyword Planner)
• Product trends shift over time; queries that were high volume in the training data may not remain so

Citation

@article{petrovic2026querylength,
  title={Is Query Length a Reliable Predictor of Search Volume?},
  author={Petrovic, Dan},
  year={2026},
  month={March},
  url={https://dejan.ai/blog/query-length-vs-volume/}
}

Author

Dan Petrovic — DEJAN AI