utahnlp / tevatron3-reranker-8b-contrastive-hf

tevatron3-reranker-8b-contrastive-hf

A pointwise passage reranker built on Qwen3-8B (dense), trained with the Tevatron 3.0 toolkit.

• Backbone: Qwen3-8B (dense)
• Training backend: HF FSDP1
• Objective: contrastive
• Parameter efficiency: full fine-tuning
• Training data: RLHN-680K

Scoring contract (important)

This checkpoint is a sequence-classification scorer , saved as Qwen3ForSequenceClassification with num_labels=1 . The relevance score of a (query, passage) pair is the single regression logit read at the last (EOS) token . Training builds the pair as "query: {{query}} passage: {{title}} {{text}}" (no yes/no suffix) and appends EOS ; score = logits[:, 0] .

This differs from the Megatron causal-LM rerankers in this release, which score logit(" yes") − logit(" no") at a prompt suffix. Load this one with AutoModelForSequenceClassification , not AutoModelForCausalLM .

Usage

Score with Tevatron's reranker eval backend (see the Tevatron repo ), or directly:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "brutusxu/tevatron3-reranker-8b-contrastive-hf"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(
    name, num_labels=1, dtype=torch.bfloat16).cuda().eval()
pair = "query: what is the capital of france passage: Paris is the capital of France."
ids = tok(pair + tok.eos_token, return_tensors="pt").to("cuda")
with torch.no_grad():
    print(model(**ids).logits[0, 0].item())

Citation

If you use this model, please cite the Tevatron toolkit.