AnkitAI / TinyJev-4B

Typed decisions, on your laptop, in one forward pass. The bigger one.

GitHub · PyPI · TinyJev 0.6B · Examples

Send this model some state, a ticket or a record or a log line, plus questions with the answers you will accept. It returns a probability for every option you offered. It cannot answer with anything else, because it never generates text; it scores the options you gave it and stops.

TinyJev 4B is the same head and the same training data as TinyJev 0.6B on a Qwen3-4B-Base backbone. It is the accurate one: 474 of 500 never-seen decisions against 440, and at confidence ≥ 0.85 it answers 87% of the queue on its own at 99.1%. It costs 8 GB (4.5 GB at 8 bits) and about 630 ms per decision on a base M1, seven times the 0.6B's.

• Choice picks one option from a list, with a probability for each.
• Noul measures whether a statement is true.
• Score places state on an ordered scale.
• Calibrated confidence (ECE 0.022), so a threshold means something and you can decide what to automate.
• A Python API, a local HTTP server, and a System One compatible endpoint.

Models

Two models so far, same head, same training data, scored on the same 500 never-seen cases from 25 domains ( benchmarks/opendecision , every case and probability logged). Latency is a base M1 (16 GB) via MLX, one forward pass per case.

Model	Params	OD-500	Gate 0.85	ms / case	Weights
TinyJev 0.6B	596M, 1.2 GB	440 (88.0%)	59% @ 98.0%	85	🤗 AnkitAI/TinyJev-0.6B
TinyJev 4B	4.0B, 8.0 GB	474 (94.8%)	87% @ 99.1%	628	🤗 AnkitAI/TinyJev-4B

OD-500 is correct answers out of 500. Gate 0.85 is the share of decisions answered on its own at confidence ≥ 0.85, and how often those were right. Calibration (ECE 0.071 vs 0.022), coverage at 2% error (63% vs 92%) and transfer-v4 dev (0.625 vs 0.762) are on the benchmark page. Load either with tinyjev.load("TinyJev-0.6B") or tinyjev.load("TinyJev-4B") .

Both rows are fp16. Loading with quantize=8 keeps the same weights in half the memory and changes almost nothing: the 0.6B scores 440 at 90 ms, the 4B 473 at 845 ms, one answer in 500 different from fp16. The gate is the number that matters in production; the rest of the queue goes to a person or a bigger model. On the same 500: Kev-0.8B 463, Claude Opus 5.5 496, the same Qwen3-0.6B weights read through letter logits with no head 354.

Measured

On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:

353/375 on dev, 121/125 on holdout, 95% CI 0.928–0.966, ECE 0.022, Brier 0.071, coverage at 2% error 92%. On Kev's transfer-v4 dev it scores 0.762 against 0.625 for the 0.6B (Kev-4B, a full fine-tune, 0.790). Every case, every probability, the coverage curves and all the baseline rows: benchmarks/opendecision .

Known weakness, measured: statement-form yes/no checks. On a 21-item check list about one support email it scores 18 (the 0.6B: 14, it says yes to everything); on hard negatives, statements about a topic the text mentions but does not support, 81%. Training data for that is the next experiment.

Use it

pip install 'tinyjev[mlx]'     # Apple Silicon
pip install 'tinyjev[torch]'   # everything else

import tinyjev
agent = tinyjev.load("TinyJev-4B", quantize=8)   # 4.5 GB in memory; drop quantize for fp16, 8 GB

agent.predict({
    "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
    "questions": {
        "team":     {"type": "choice", "instructions": "Which team should handle this?",
                     "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
                                  "shipping": "Delivery status, delays, lost packages",
                                  "billing":  "Charges, invoices, payment problems"}},
        "escalate": {"type": "noul",   "instructions": "Does this need urgent human attention?"},
        "anger":    {"type": "score",  "instructions": "How angry is the customer?",
                     "criteria": ["calm", "frustrated", "very angry"]},
    }})

Eight bits changed one answer in 500 on the held-out suite. Serve it over HTTP, speaking the System One request shape:

tinyjev serve TinyJev-4B --quantize 8     # POST /v1/systemone on 127.0.0.1:8077

What is in this repo

AutoModel.from_pretrained("AnkitAI/TinyJev-4B") loads the backbone on its own, a standard Qwen3Model in fp16 with the LoRA already merged. The decision head lives in head.safetensors , and tinyjev is what turns hidden states into calibrated answers.

How it was built, and how it scores

Qwen3-4B-Base with a pointer head, LoRA r16 at lr 5e-5, seed 2, two epochs on the public jaredpalmer/kev-suites decision-v7 training split (12,576 records), Kev's study runner on one H100 for 45 minutes, the adapter merged back into the base. No held-out transfer source was used in training. Temperature 1.0 at inference; the calibration figures above are at raw logits.

Support the Project

If this model is useful in your work, you can support independent research:

Credits

Built on Qwen3-4B-Base (Apache-2.0). The training data, evaluation suites and the pointer-head design come from Kev by Jared Palmer (Apache-2.0). The typed-decision interface follows TypeSafe's Jev . MIT licensed.