FINAL-Bench / Darwin-27B-ZTC-v5

Darwin-27B-ZTC-v5

Darwin-27B-ZTC-v5 is a decision model. It does not generate text. Given a state and a typed question, it returns a calibrated probability for every option in one forward pass: yes or no ( noul ), one of several options ( choice ), or a level on an ordered scale ( score ). Requests use the System One shape ( state plus questions ), so clients written for that format work unchanged.

It is the next version of Darwin-27B-ZTC-v2 .

Results

S1MB (System One Mosaic Benchmark), english-v1

All 137 benchmarks complete, measured with the S1MB evaluator and its unchanged autojev adapter. The numbers follow the leaderboard's own code ( viewer/src/lib/borda.ts ) applied to the published results (150 models, 2026-10-11) with this model added. Submission pending maintainer review; see the leaderboard .

Typed Decisions test (LocalLLaMA/typed-decisions, 2,000 decisions)

Training and data

Full-weight fine-tuning of Darwin-27B-ZTC-v2 on public train splits of decision datasets (including the Typed Decisions train split) under licenses that allow commercial use. No test split of S1MB, Typed Decisions or any other benchmark was used for training, and every training row was screened against the S1MB english-v1 test cases.

Usage

Python 3.12, a CUDA GPU with about 60 GB of free memory (BF16 weights), and the code shipped in this repository ( autojev/ ).

import sys
from pathlib import Path
from huggingface_hub import snapshot_download

path = Path(snapshot_download("FINAL-Bench/Darwin-27B-ZTC-v5"))
sys.path.insert(0, str(path))
from autojev.model import DecisionModel, answer

model = DecisionModel(checkpoint=path).eval()
question = {
    "type": "noul",
    "instructions": "Does this message need a reply today?",
    "criteria": {"true": "Time-sensitive", "false": "Can wait"},
}
row = {"state": "Our payouts have failed for three days.", "question": question}
print(answer(question, model.predict([row])[0]))

ztc_server.py serves POST /v1/systemone with batching.

Serve with transformers serve

The repository also runs on the standard /v1/systemone endpoint of transformers serve (a transformers build that includes /v1/systemone , main branch as of October 2026). systemone_config.json holds the prompt template, answer labels and temperature.

hf download FINAL-Bench/Darwin-27B-ZTC-v5 systemone_config.json --local-dir .
transformers serve FINAL-Bench/Darwin-27B-ZTC-v5 --decision-config systemone_config.json

curl http://localhost:8000/v1/systemone   -H "Content-Type: application/json"   -d '{"model": "FINAL-Bench/Darwin-27B-ZTC-v5",
       "state": "I was charged twice for my order and want a refund.",
       "questions": {
         "refund": {"type": "noul", "instructions": "Is a refund requested?"},
         "intent": {"type": "choice", "instructions": "What does the customer want?",
                    "criteria": {"refund": "wants money back", "cancel": "wants to cancel an order", "track": "wants to track an order"}}}}'

License

Apache-2.0. The decision code in autojev/ is MIT (upstream autojev by Denis Yarats) with changes for text-only backbones.