DJLougen / eljefe-v0

ElJefe v0 — Local-vs-Frontier Router

ElJefe is a 22M-parameter MiniLM encoder with two heads that predicts, for a given prompt, whether a local Gemma-4-E4B answer is sufficient or whether the request is worth escalating to a frontier model . ElJefe does not answer the user's question — it estimates the marginal value of additional compute.

• Head 1 (classification): p_local — probability the local model passes at acceptable quality.
• Head 2 (regression): delta_q — expected quality gain from the frontier model.

Quickstart

pip install onnxruntime transformers huggingface_hub

import onnxruntime as ort
from transformers import AutoTokenizer
from huggingface_hub import hf_hub_download

tok = AutoTokenizer.from_pretrained("DJLougen/eljefe-v0", subfolder="tokenizer")
sess = ort.InferenceSession(
    hf_hub_download("DJLougen/eljefe-v0", "onnx/router_fp32.onnx"))

enc = tok(["How many r's are in strawberry?"], return_tensors="np",
          truncation=True, max_length=256)
p_local, delta_q = sess.run(None, dict(enc))   # outputs: p_local, delta_q
route = "local" if p_local[0] >= 0.9 else "frontier"

PyTorch weights are in model.pt (state dict of the two-head module); router.pkl is the pickled eljefe.router.Router wrapper for the training repo .

Choosing a threshold

The delta_q head is a second signal: escalate when delta_q > epsilon even if p_local is middling — useful when a bad local answer is costly.

Training data

Trained on 9,553 counterfactual pairs from DJLougen/eljefe-router-data : each prompt was answered by google/gemma-4-E4B-it (local, Colab L4) and deepseek-v4-flash-0731 (frontier, Fireworks), then graded deterministically (exact-match / numeric / code-tests / constraint checks — no LLM judge).

Labels: local_sufficient = local_score >= 0.70 AND delta_q <= 0.10 .

Results (test_iid, n=995)

What "oracle" means: the oracle is a cheat-mode router computed after the fact. Since both models ran on every prompt, we know each row's actual local_score and frontier_score ; the oracle picks the better route per row (local whenever local_score >= frontier_score ). It's not deployable — it "knows" the answer before choosing — but it marks the ceiling: the best possible cost-quality tradeoff any router could achieve on this data.

At threshold 0.6 ElJefe keeps 62% of traffic local at 99.6% quality retention . ROC-AUC 0.861 vs 0.609 for the heuristic.

Frontier-swap transfer

Evaluated against counterfactuals from two other frontier models (glm-5p3-flash, gpt-oss-120b) it was never trained on, ElJefe retains ~95% of oracle utility — the learned boundary generalizes across escalation targets.

Limitations

• Trained only on objectively-gradable tasks (math, code, MCQ, instruction following). Open-ended/chat quality is unvalidated.
• Binary graders make local_sufficient frontier-independent; the delta_q head carries the frontier-specific signal.
• Scores reflect canonical bf16 E4B on an L4 — a quantized local build may shift the boundary (see scripts/16_mac_calibration.py in the repo).

Reproduce

Full pipeline (fetch → dual generation → deterministic grading → splits → train → calibrate → evaluate → ONNX): https://github.com/DJLougen/ElJefe — see README and PLAN_ElJefe_Colab_CLI.md .