caiovicentino1 / Eikos-27B

Eikos-27B — open typed-decision model (System-1)

Eikos-27B answers typed decisions about a given state in a single forward pass , with calibrated probabilities over a bounded set of options. It does not generate text at decision time.

• Question types: noul / boolean (yes/no with probability), choice (one of N options) and score (ordinal levels).
• Outputs: the full probability distribution plus a confidence value, so callers can abstain below a threshold.
• Focus: global finance, trading and trade finance.
  • Apply a stated rule, policy or rulebook to a case: order limits, margin, wash sales, price bands, payments limits, KYC/AML, lending policies, documentary credits, Incoterms® stage mapping, VAT/GST.
  • Entity-level financial sentiment, answer verification over financial tables, temporal and numeric checks.
• Scope limit: it applies rules to the facts it is given. It does not predict prices.
• Languages: trained in English and Portuguese. Spanish was held out entirely and is used as a zero-shot test.
• Runs anywhere: one self-contained checkpoint in the official Qwen layout, with no adapters, no external LLM and no API.
  • vLLM, with prefix caching for many questions over the same state and for agent sessions;
  • PyTorch / transformers;
  • Apple Silicon via MLX or MPS.

Eikos (εἰκός, "the probable" ) comes in two sizes: Eikos-4B and Eikos-27B (this model), each with FP8 and INT4 builds for GPUs and MLX builds for Apple Silicon (4B: 8-bit and 4-bit; 27B: 4-bit). Training data: caiovicentino1/eikos-decisions . Code: https://github.com/caiovicentino/eikos .

Numbers below are from our own evaluation harness (see Evaluation ). The official JevBench leaderboard submission is pending.

How it works

1. Prompt. The state (evidence), the question (criterion) and the lettered options go into a structured prompt, following the SemIf format.

2. Readout. The model reads the next-token logits only over the option letters . A softmax with a fixed temperature gives the distribution.

3. Calibration. Temperature T = 1 ships by default (see Calibration ).

4. Training. Distillation from a strong teacher (GLM-5.3-Flash, maximum reasoning effort) on:

   • generated and blindly labeled decision items;
   • exact programmatic items;
   • compositional rules;
   • long-context dossiers (6k–12k tokens).

   Training-only signals, all switched off at inference:

   • soft cross-entropy to the teacher's probabilities, with option-order permutation;
   • an auxiliary loss on the item's short rationale (written with the item, explaining the gold answer).

   Eikos-27B is a single LoRA fine-tune of Qwen3.8-27B (rank 64, one epoch) on the same data as Eikos-4B, with long-context dossiers up to 12k tokens and without the EN↔PT view objective.

Quick start

vLLM (production: batching + prefix cache)

Requires vLLM ≥ 0.30.0. Older builds return wrong answers when several long requests are batched together on this hybrid (Gated DeltaNet) architecture; we measured drops of 3–6 points on long, shared-document items with vLLM 0.11. With 0.30 the batched results match the PyTorch reference.

bash serve_vllm.sh <MODEL_DIR> 8001      # vLLM engine: letter readout + hybrid prefix cache (checks vLLM >= 0.30)
python serve.py --model <MODEL_DIR> --vllm-url http://127.0.0.1:8001 --port 8000   # TypeSafe-compatible API on :8000

serve_vllm.sh runs vllm serve with the flags the readout needs: --served-model-name decider , --enable-prefix-caching --mamba-cache-mode all , --logprobs-mode processed_logprobs --max-logprobs 32 , --dtype bfloat16 and --max-model-len 16384 .

HTTP API (TypeSafe-compatible)

curl -s localhost:8000/v1/systemone -d '{
  "state": "Order ticket #A-2231. Retail client, moderate risk. BUY 1,500 XYZ at market. Equity USD 48,000. Last price USD 41.20. Rule 4.2: a single order may not exceed 50% of equity without written supervisor approval. Approvals on file: none.",
  "questions": {
    "allowed": {"type": "noul", "instructions": "Under rule 4.2, can this order be executed as submitted?",
                "criteria": {"true": "complies with rule 4.2", "false": "breaches rule 4.2"}},
    "action":  {"type": "choice", "instructions": "What should the desk do?",
                "criteria": {"execute": "send as submitted", "request_approval": "hold and ask a supervisor",
                             "reduce_size": "cut the order to the allowed size", "reject": "refuse the order"}},
    "risk":    {"type": "score", "instructions": "How risky is this ticket for the firm?",
                "criteria": ["low", "moderate", "high", "critical"]}
  }}'

• One pass for the whole request: all questions are answered together, and the shared state is processed once through the prefix cache.
• Response per question:
  • noul : probability (of "yes"), value and confidence ;
  • choice : choice , probabilities and confidence ;
  • score : score , probabilities , expected and confidence .

Agent sessions (incremental state + cache)

POST /v1/sessions                 {"state": "..."}          -> {"session_id"}
POST /v1/sessions/<id>/append     {"text": "..."}           -> grows the state
POST /v1/sessions/<id>/systemone  {"questions": {...}}      -> answers over the current state
DELETE /v1/sessions/<id>

On a hybrid (linear-attention) model the state is cached at the end of the context and forked for every question. In our tests a 15-step agent session gave answers identical to sending the full request each time.

Local (Apple Silicon)

python mlx_decide.py <MODEL_DIR>       # MLX backend: same readout and calibration as PyTorch
python local_demo.py <MODEL_DIR> mps   # PyTorch MPS

Evaluation

All numbers are our own measurements with a single harness. We never trained on any evaluation set. Spanish, one task family (trade-offs) and one topic (healthcare administration) were held out of training entirely.

* generated by the same rule generator family used in training (different rules and cases); see notes.

How to read the table:

• JevBench: public items only, not the official leaderboard run.
• Jev: measured through its API on our batteries, not distilled from.
• Laya (charts only): run with its official package; its JevBench scores are within one item of its official leaderboard results.
• Charts: 7,140 items common to all four systems. The confidence chart shows, for every confidence threshold, how many decisions a system would take on its own and how often those decisions are wrong.
• WCB (central-bank stance): reported as balanced accuracy, because the classes are imbalanced.
• Rules suites: "rules / new domain / rulebooks" come from the same generator family used in training, so they measure learning of that generator. Transfer to rules the model never saw is measured by "Trade (unseen rules)" (Incoterms®, documentary-credit presentation, EU VAT) and by RuleArena.
  • "New domain" has two parts. 492 items come from a domain absent from training (insurance): 27B 95.1, 4B 91.5. The other 108 items use a rule combination held out of training (an exception over a business-day window): 27B 91.7, 4B 90.7. That combination appears in only 6 of the ~6,000 rule rows used in training.

Long context

The case file is hidden inside unrelated text at the start, middle or end of the prompt (80 public JevBench decisions per point; released checkpoints served with vLLM 0.30).

Release builds

Final evaluation of every published build, run on the exact files in these repos: 7 suites, 7,371 items, vLLM 0.30 with batching and prefix cache on (MLX builds: MLX's CUDA backend on Linux). Batched vLLM is not bit-for-bit deterministic across runs (1–2 items per suite can change), so the bf16 rows differ slightly from the Evaluation table above, which uses our reference harness. "≥0.90" = share of decisions the model would take on its own at ≥90% confidence, and the real error rate among them.

Release gate , fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.

• Quantization cost: no build loses more than 2 points on any metric of any suite, and calibration (ECE) stays within 0.01 of bf16 or better.
• Eikos-4B-INT4 keeps accuracy and calibration but changes 4.5% of answers, above the 3% the gate allows. Almost all of them (94%) are on items where bf16 itself was unsure (confidence below 0.7); on decisions bf16 takes with confidence ≥0.9, it gives the same answer on 100% of items. We publish it with this note.
• Mac builds (MLX): Eikos-4B-MLX-8bit passes the gate. Eikos-27B-MLX-4bit and Eikos-4B-MLX-4bit keep accuracy and calibration but change 3.4% and 7.5% of answers. Most of those changes (92% and 93%) are on items where bf16 itself was unsure; on decisions bf16 takes with confidence ≥0.9, they agree on 99.9% and 99.8% of items. We publish them with this note. The small JevBench-original split (72 items) moves by a few items between builds.

Calibration

• Why T = 1: our held-out calibration sets turned out easier than real hard data. Every temperature fitted on them was below 1 and made the model over-confident on hard items. Absent a realistic calibration set, we ship T = 1.
• Changing it: calib.json holds the temperature, and callers can re-fit it on their own data.

Parallelism and speed

• Same state, many questions: the hybrid prefix cache processes the state once.
  • 200 questions over a 3.3k-token state ran 21× faster with vLLM prefix caching than without it.
• Throughput: batched decisions reach ~51 decisions/s on a shared GPU, with results identical to PyTorch.
• Full GPU (RTX PRO 6000, vLLM, hybrid prefix cache), 200 questions over one state:
  • 4B: 161 decisions/s on a 751-token state and 197 decisions/s on a 3.3k-token state;
  • 27B: 44 decisions/s on a 3.3k-token state.
• Our own PyTorch cache path: 1,500 questions over one state in 8.7 s (4B).
• Apple M4 16 GB, MLX bf16: ~0.4–0.8 s per decision, and 3 questions over one state in ~1.1 s.

Limitations

• Single pass means no multi-step reasoning. Long chains of arithmetic across many rules are out of reach; on RuleArena (NBA salary-cap rulebooks, ~25k tokens) the model does not discriminate. An optional "verify" mode (a short reasoning budget before the letter) exists in the server; it is off by default and not part of any reported number.
• General world knowledge is bounded by model size. Both sizes trail large frontier systems on knowledge-heavy tasks such as MMLU-Pro.
• Monetary-policy stance (hawkish/dovish) is a known weakness of both sizes: balanced accuracy 38.4 (4B) and 44.5 (27B), against 58.6 for Jev.
• Context: trained on inputs up to 32k tokens (4B) or 12k (27B). The base supports longer inputs, but accuracy beyond the trained range degrades gradually.
• Not advice: it is not legal, tax or investment advice. It applies the rules it is given and does not know your jurisdiction's current law.

Training data and licenses

See NOTICE .

• Model and code: MIT for our contributions. The base model is Apache-2.0 ( LICENSE-Qwen ).
• Third-party data, with attribution: FinEntity (ODC-BY 1.0), TAT-QA (CC BY 4.0), GSM8K train (MIT).
• Prompt format: SemIf (MIT).
• Trademark: Incoterms® is a trademark of the ICC. No ICC or regulator text was used.

Citation

@misc{eikos2026,
  title  = {Eikos: open, calibrated, single-pass typed-decision models for finance and trading},
  author = {Caio Vicentino},
  year   = {2026},
  url    = {https://github.com/caiovicentino/eikos}
}