eulogik / flashrank-pro-base

FlashRank-Pro Base

Direct answer

FlashRank-Pro Base is a 149M-parameter ModernBERT cross-encoder that reranks candidate passages for English RAG. It takes one query plus one candidate document, encodes them jointly, and returns a relevance score between 0 and 1. Use it after BM25 or dense retrieval when you need better top-5 or top-10 precision without a multi-billion-parameter model.

One-line story: a small bidirectional reranker learned to separate hard MS MARCO negatives, then improved the weakest retrieval domains the most.

Find this model with: modernbert reranker , cross encoder rerank , rag reranker , beir ndcg at 10 , cpu friendly reranker , apache 2.0 reranker , 149m reranker .

When to use it

Quick start

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_id = "eulogik/flashrank-pro-base"
model = AutoModelForSequenceClassification.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
model.eval()

query = "how to train a neural network"
documents = [
    "Training neural networks requires backpropagation and gradient descent.",
    "Python is a programming language.",
    "The history of ancient Rome is long and complex.",
]

pairs = [query + " </s> " + document for document in documents]
inputs = tokenizer(pairs, truncation=True, max_length=256, padding=True, return_tensors="pt")

with torch.no_grad():
    scores = torch.sigmoid(model(**inputs).logits.float().squeeze(-1)).tolist()

ranked = sorted(zip(documents, scores), key=lambda item: item[1], reverse=True)
print(ranked)

Benchmarks

Evaluation protocol

• Text-only BM25 top-100 retrieval, then rerank with the cross-encoder.
• max_length=256 , query-batched evaluation.
• webis-touche2020 was omitted because its 382K-document corpus exceeded Colab RAM.
• Baseline values are the same training run's initialization checkpoint, recorded in the project session log.
• These are not official BEIR leaderboard numbers. Retrieval text fields, tokenization, candidate count, and hardware can change absolute scores.

Architecture

The important choice is bidirectionality: query and document tokens attend to each other in the same encoder pass. That is why a 149M cross-encoder can be competitive as a precision layer even though it is much smaller than decoder-only rerankers.

Training recipe

Limitations

• English-focused; multilingual quality was not measured.
• The reported evaluation is a five-set BEIR-style reranking protocol, not the full official BEIR suite.
• Absolute NDCG@10 depends on the retriever, text fields, candidate count, tokenizer length, and hardware.
• Latency, throughput, memory, and energy were not benchmarked in this run; treat “CPU-friendly” as a design goal, not a measured guarantee.
• Sigmoid outputs are ranking scores, not calibrated probabilities.
• Touché-2020 was excluded for memory reasons rather than quality reasons.

FAQ

What is FlashRank-Pro Base?

It is an Apache 2.0 ModernBERT cross-encoder for reranking English passages in retrieval-augmented generation.

How many parameters does it have?

Approximately 149M parameters, based on ModernBERT-base plus a scalar relevance head.

What data trained it?

100,000 MS MARCO examples with one positive passage and six hard negatives per query.

How is it different from an embedding model?

An embedding model retrieves candidates independently. This cross-encoder jointly reads one query and one candidate, making it slower but more precise for final ranking.

Can I use it commercially?

Yes. The model card license is Apache 2.0. Verify the repository license file before production deployment.

How should I deploy it?

Retrieve 50-200 candidates with BM25 or dense search, rerank them in batches, and keep the top 5-20 for generation.

What should I measure before trusting it?

Measure NDCG@10 on your own queries, latency per candidate batch on your CPU/GPU, worst-case long-document behavior, and calibration if scores affect decisions.

Reproduction notes

• Training entry point: training/05_beir_finetune.py
• Training notebook: notebooks/flashrank_beir_finetune_colab.ipynb
• Data preparation: scripts/prepare_msmarco_data.py
• Project decisions and session history: MEMORY.md

License

Apache 2.0. Built on ModernBERT by AnswerDotAI.

Share line: FlashRank-Pro Base: a 149M Apache-2.0 ModernBERT reranker for RAG. Measured 0.3314 five-set NDCG@10 and +22.6% versus its run-init checkpoint. #RAG #reranker #ModernBERT #BEIR #opensource