Falconsai / LightDec

Source model card: Falconsai/LightDec @ main , carried verbatim below. Its licence is the repository's. The Model Surgeon record follows it.

Falconsai/LightDec

View in Model Surgeon

A lightweight, single-pass, typed, calibrated decision model for agentic systems. Give it a state (text, code or JSON), one or more typed questions ( choice , noul yes/no, score ordinal) and a closed set of options. It returns a calibrated probability for every option, from one encoder pass per question.

LightDec is the FalconDec architecture trained with FalconDec notebook V2 on the standard data preset. That run adds agent-specific decisions (AgentTrek, Counsel, HotpotQA) to a balanced mix of 58 test tasks across 9 domains. This page is both the model card and the developer guide : how to load it, test it, and use it as a decision component in an agentic system.

At a glance. Test accuracy 0.725 (micro and task-macro) on 17,498 decisions from 58 tasks, with ECE 0.025 . At a 0.70 confidence threshold, LightDec answers 56% of decisions at 89.6% accuracy and defers the rest. Weights: 319 MB fp16, 161 MB int8. It is strongest on support routing, code understanding, intents, guardrails and agent-step checks, and weakest on multi-step arithmetic, date and table reasoning, and very wide label sets. Evaluate it on your own traffic (§6.4) before acting on its answers.

Contents

1. Model summary
2. Results
3. Intended and out-of-scope uses
4. The Hub repository
5. Install, load and read a result
6. Testing the model
7. Using it in an agentic system
8. Tuning for your domain
9. Architecture
10. Training data
11. Training procedure and calibration
12. Operational notes
13. Bias, risks and limitations
14. Versioning and lineage
15. API reference
16. Citation and references

---

1. Model summary

The model has no generative component. It can only rank the options you give it, so it cannot produce text outside that set.

---

2. Results

All numbers come from this checkpoint's falcondec_report.json : one run, standard preset, 2 epochs, seed 42, MODE="scratch" , notebook V2.

2.1 At a glance

Selective prediction is the headline. Calibration is good (ECE 0.025), so the confidence score is a reliable gate. Acting only on decisions with confidence ≥ 0.70 covers 56% of traffic at 89.6% accuracy, against 72.5% accuracy when answering everything. That is the property an agent loop needs: answer the easy majority locally, and hand the rest to an LLM or a human.

2.2 Per domain

2.3 Per task

Held-out tasks were never used for training, calibration or model selection. "proof_v2 (card)" lists proof_v2's published score for the same source and task (different samples; indicative only).

2.4 Comparison with proof_v2 (indicative)

On the 22 tasks that both this report and proof_v2's model card cover, LightDec's task-macro accuracy is 0.807 vs 0.679 , and it scores higher on 20 of 22 . The largest gains are on the tasks proof_v2 reported as weak:

LightDec is lower on CLINC150 (0.793 vs 0.850) and MBPP task→solution (0.987 vs 0.992).

These are different test samples and, for some tasks, different question formats . For example, LightDec's MultiNLI task is three-way NLI, and its bug-spotting mutants are verified to fail the unit tests. The like-for-like head-to-head, which runs proof_v2 on identical decisions (notebook cell 23), did not run for this checkpoint ( head_to_head: null ).

2.5 Comparison with Laya and TypeSafe Jev (indicative)

Laya's numbers are from its own benchmark report; Jev's are third-party published. LightDec was trained on the typed-decisions training split, like the fine-tuned Laya checkpoint. On typed-decisions LightDec trails both (the teacher-agreement ceiling is 0.735 and the majority-class baseline 0.461). It matches Jev on Emotion, edges Laya on all-77 Banking77, and trails both on AG News. LightDec is 2.6× smaller than Laya's 421M English checkpoint.

---

3. Intended and out-of-scope uses

Intended

Out of scope

• Multi-step arithmetic and quantitative reasoning. AQuA 0.259 (chance 0.20); counting and summing policies 0.52; comparing values in a table 0.290. GSM8K reaches 0.637 only because it is posed as 4-option multiple choice with near-miss distractors. Route real math to an LLM or code.
• Date reasoning in unfamiliar formats. The invoice-overdue transfer test (ISO dates, whereas training used "Month DD, YYYY") scores 0.407, below chance, with ECE 0.364 . It is confidently wrong there. Normalise dates before asking, or compute them in code.
• Very wide label sets. All 77 Banking77 intents in one question score 0.470. Pre-filter to a shortlist of about 20 options (§7.1).
• Hard commonsense and exam knowledge : HellaSwag 0.383, ANLI 0.393, MMLU 0.393.
• Code security and correctness gating : Devign 0.553 is near chance. Don't use it to approve code.
• Open-ended questions : it always picks one of your options. Add "None of the above" when appropriate; it was trained with that option.
• Non-English text , and high-stakes decisions without human oversight .

---

4. The Hub repository

Pin a revision in production. The repo can change, so pass a commit hash when you load.

---

5. Install, load and read a result

pip install torch "transformers>=4.48" safetensors huggingface_hub numpy

5.1 Load

Save this helper as lightdec.py next to your code. Every example below uses it.

# lightdec.py
import importlib.util, json, shutil
from pathlib import Path
from huggingface_hub import snapshot_download


def load_lightdec(repo="Falconsai/LightDec", revision=None, variant="fp16", device=None, dtype=None):
    """Returns (fdm, model, tokenizer). variant: "fp16" (319 MB) or "int8" (161 MB, dequantised on load)."""
    path = Path(repo) if Path(repo).exists() else Path(snapshot_download(repo, revision=revision))
    if variant == "int8":
        path = path / "compact-int8"
    fc = json.loads((path / "falcondec_config.json").read_text(encoding="utf-8"))
    expected = fc.get("weights", "model.safetensors")
    if not (path / expected).exists():                  # e.g. a renamed weight file in a processed copy
        cands = sorted(path.glob("*.safetensors"))
        if not cands:
            raise FileNotFoundError(f"no .safetensors weights in {path}")
        local = Path("lightdec_local") / variant
        shutil.copytree(path, local, dirs_exist_ok=True)
        shutil.copy(cands[0], local / expected)
        path = local
    spec = importlib.util.spec_from_file_location("falcondec_modeling", str(path / "falcondec_modeling.py"))
    fdm = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(fdm)
    model, tok = fdm.load_falcondec(str(path), device=device, dtype=dtype)   # cuda if available, else cpu
    return fdm, model, tok

from lightdec import load_lightdec

fdm, model, tok = load_lightdec()                      # or load_lightdec(revision="<commit>", variant="int8")
print(model.fcfg["name"], model.fcfg["version"], round(model.num_parameters() / 1e6, 1), "M params")

If loading prints [FalconDec] load warning: missing=… unexpected=… , the weights didn't match the architecture. Treat that as a failed load (§6.1 checks for it).

5.2 Decide

decide() takes one state and any number of typed questions: either a list, or a Jev/Laya-style dict keyed by name.

state = {"from": "[email protected]", "subject": "Duplicate charge on invoice #4411",
         "body": "We were billed twice for March. Please refund the duplicate today or we will cancel our plan."}

out = fdm.decide(model, tok, state, {
    "department": {"type": "choice", "instructions": "Which department should handle this request?",
                   "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
                                "sales": "pricing, contracts", "other": "everything else"}},
    "urgency":    {"type": "score", "instructions": "How urgent is this request?",
                   "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
})
a = out["answers"]
print(a["department"]["choice"], round(a["department"]["confidence"], 3), a["department"]["defer"])
print("urgency level", round(a["urgency"]["expected_level"], 2), "of", 2)
print("P(churn)", round(a["churn_risk"]["p_true"], 3))

Plain options work too: {"question": "Which team?", "options": ["Accounts", "Billing", "Shipping"]} .

5.3 Reading a result

Each item in out["results"] (and out["answers"][key] ) contains:

How to read them:

• Low confidence, spread probabilities : the state doesn't support any option clearly. Defer, or add "None of the above".
• noul near 0.5 : genuinely ambiguous. Ask for more information rather than guessing.
• score : use expected_level for thresholds ("escalate if ≥ 1.5") rather than choice .

---

6. Testing the model

Tests 6.1–6.3 need no labelled data, so run them in CI whenever you change the revision. Test 6.4 is the one that tells you whether to ship.

6.1 Integrity and determinism

Save as check_lightdec.py and run python check_lightdec.py [revision] .

import contextlib, io, sys
import numpy as np
from lightdec import load_lightdec

rev = sys.argv[1] if len(sys.argv) > 1 else None
log = io.StringIO()
with contextlib.redirect_stdout(log):
    fdm, model, tok = load_lightdec(revision=rev)
assert "load warning" not in log.getvalue(), log.getvalue()

fc = model.fcfg
assert fc["name"] == "FalconDec", fc["name"]           # LightDec checkpoints use the FalconDec architecture name
T = model.temperature.float().cpu().numpy()
assert T.shape == (3, 4) and (T > 0).all(), T
q = {"team": {"question": "Which team?", "options": ["recover password", "shipping", "invoicing"]}}
a = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
b = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
assert all(abs(a["probs"][k] - b["probs"][k]) < 1e-4 for k in a["probs"]), "non-deterministic"
assert abs(sum(a["probs"].values()) - 1) < 1e-3
print(f"OK  LightDec (FalconDec v{fc['version']}, notebook {fc.get('notebook_version')})  choice={a['choice']}  "
      f"conf={a['confidence']:.3f}  defer_threshold={fc.get('defer_threshold')}")

The stored temperatures should read approximately [[1.707, 1.352, 1.466, 1.349], [1.402 ×4], [1.453 ×4]] (§11.2).

6.2 Behavioural tests (pytest)

Save as tests/test_lightdec.py and run pytest -q . Set LIGHTDEC_REVISION to test a pinned commit.

import os, random
import pytest
from lightdec import load_lightdec


@pytest.fixture(scope="session")
def fd():
    return load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"))


def ask(fd, state, question, options, **kw):
    fdm, model, tok = fd
    return fdm.decide(model, tok, state, [dict(question=question, options=options, **kw)])["results"][0]


def test_probabilities_are_valid(fd):
    r = ask(fd, "The build failed on main.", "What next?", ["Revert", "Ignore", "Retry"])
    assert all(0 <= p <= 1 for p in r["probs"].values()) and abs(sum(r["probs"].values()) - 1) < 1e-3


def test_typed_outputs(fd):
    fdm, model, tok = fd
    out = fdm.decide(model, tok, "I was charged twice. Refund me or I'm leaving.", {
        "refund": {"type": "noul", "instructions": "Does the user ask for a refund?"},
        "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]}})["answers"]
    assert 0 <= out["refund"]["p_true"] <= 1 and out["refund"]["choice"] in (True, False)
    assert 0 <= out["urgency"]["expected_level"] <= 2


def test_support_routing(fd):
    r = ask(fd, "I forgot my password and the reset email never arrived.", "Which team should handle this?",
            ["recover password", "billing and payment", "delivery information"])
    assert r["choice"] == "recover password"


def test_fanout_matches_single_questions(fd):
    # Batching changes padding; under bf16 that moves probabilities slightly, never the substance.
    fdm, model, tok = fd
    state = "I forgot my password and can't sign in."
    qs = [{"question": "Team?", "options": ["recover password", "shipping", "invoicing"]},
          {"question": "Urgent?", "options": ["Yes", "No"]}]
    together = fdm.decide(model, tok, state, qs)["results"]
    for q, t in zip(qs, together):
        alone = fdm.decide(model, tok, state, [q])["results"][0]
        assert all(abs(alone["probs"][k] - t["probs"][k]) < 2e-2 for k in alone["probs"])


def test_option_order_is_mostly_irrelevant(fd):
    # The head is order-equivariant, but the encoder sees positions; training reshuffled options every epoch.
    state, q = "Where is my parcel? It's three days late.", "What should support do?"
    opts = ["Give the delivery status", "Start a refund", "Book an appointment"]
    base = ask(fd, state, q, opts)["choice"]
    same = sum(ask(fd, state, q, random.Random(s).sample(opts, len(opts)))["choice"] == base for s in range(5))
    assert same >= 4


def test_many_options_use_the_tournament(fd):
    opts = [f"topic number {i}" for i in range(119)] + ["reset my password"]
    r = ask(fd, "I can't log in, I need to reset my password.", "What does the user want?", opts)
    assert len(r["probs"]) == 120 and abs(sum(r["probs"].values()) - 1) < 1e-3


def test_int8_agrees_with_fp16(fd):
    fdm8, m8, tok8 = load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"), variant="int8")
    fdm, model, tok = fd
    items = [dict(state=s, question="Which team?", options=["billing", "shipping", "accounts", "technical"])
             for s in ["I was double charged", "Where is my parcel?", "Change my email", "The app crashes on start",
                       "Refund the duplicate payment", "Package never arrived", "Reset my login", "Error 500 on checkout"]]
    a = [p.argmax() for p in fdm.score_items(model, tok, items)]
    b = [p.argmax() for p in fdm8.score_items(m8, tok8, items)]
    assert sum(x == y for x, y in zip(a, b)) >= len(items) - 1

6.3 Latency

import time, numpy as np, torch
from lightdec import load_lightdec

fdm, model, tok = load_lightdec()
q = [{"question": "Route?", "options": ["billing and payment", "shipping", "recover password"]}]
for _ in range(5):
    fdm.decide(model, tok, "I was charged twice.", q)
t = []
for _ in range(100):
    if torch.cuda.is_available(): torch.cuda.synchronize()
    t0 = time.perf_counter(); fdm.decide(model, tok, "I was charged twice.", q)
    if torch.cuda.is_available(): torch.cuda.synchronize()
    t.append((time.perf_counter() - t0) * 1000)
print(f"p50 {np.percentile(t, 50):.1f} ms   p95 {np.percentile(t, 95):.1f} ms   on {model.device}")

For CPU serving, load the int8 variant with device="cpu", dtype=torch.float32 , and optionally apply torch.ao.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8) for int8 matrix multiplies.

6.4 Accuracy on your own labelled data

Write 50–500 decisions that look like your real traffic, one JSON object per line. expected may be a letter, a 0-based index or the option text; type is optional.

{"id": "t1", "tag": "support", "state": "…", "question": "…", "options": ["…", "…"], "expected": "B", "type": "choice"}

import json, string
import numpy as np
from lightdec import load_lightdec

fdm, model, tok = load_lightdec()
rows = [json.loads(l) for l in open("my_eval.jsonl", encoding="utf-8") if l.strip()]

def idx(v, opts):
    if isinstance(v, int): return v
    v = str(v).strip()
    if len(v) == 1 and v.upper() in string.ascii_uppercase[:len(opts)]: return string.ascii_uppercase.index(v.upper())
    return [o.lower() for o in opts].index(v.lower())

items = [dict(state=r["state"], question=r.get("question", ""), options=r["options"], type=r.get("type", "choice"))
         for r in rows]
probs = fdm.score_items(model, tok, items, batch_size=64)
gold = np.array([idx(r["expected"], r["options"]) for r in rows])
pred = np.array([p.argmax() for p in probs]); conf = np.array([p.max() for p in probs]); ok = pred == gold

def ece(c, k, bins=15):
    e = 0.0
    for lo in np.linspace(0, 1, bins, endpoint=False):
        m = (c > lo) & (c <= lo + 1 / bins)
        if m.any(): e += m.mean() * abs(c[m].mean() - k[m].mean())
    return e

print(f"accuracy {ok.mean():.3f} | ECE {ece(conf, ok):.3f}")
for tag in sorted({r.get("tag", "all") for r in rows}):
    m = np.array([r.get("tag", "all") == tag for r in rows]); print(f"  {tag:12s} n={m.sum():4d} acc={ok[m].mean():.3f}")
for t in (0.5, 0.6, 0.7, 0.8, 0.9):
    m = conf >= t
    print(f"  act if conf >= {t}: answers {m.mean():6.1%}, accuracy when answering {ok[m].mean() if m.any() else float('nan'):.3f}")

The last loop is the deferral policy of §7.3; on the published test mix, 0.70 gives 56% coverage at 0.896 accuracy. Pick the smallest threshold whose "accuracy when answering" meets your bar. If ECE on your data is much higher than 0.025, recalibrate (§8.2).

6.5 Regression gate between revisions

Fail the pipeline if any test task drops by more than two points between two revisions:

import json, sys
from huggingface_hub import hf_hub_download

old_rev, new_rev = sys.argv[1:3]
rep = lambda rev: {r["task"]: r["acc"] for r in json.load(open(
    hf_hub_download("Falconsai/LightDec", "falcondec_report.json", revision=rev), encoding="utf-8"))["test_per_task"]}
old, new = rep(old_rev), rep(new_rev)
bad = [(t, old[t], new[t]) for t in new if t in old and new[t] < old[t] - 0.02]
print("\n".join(f"REGRESSION {t}: {a:.3f} -> {b:.3f}" for t, a, b in bad) or "no regressions")
sys.exit(1 if bad else 0)

---

7. Using it in an agentic system

7.1 Where it fits

An agent loop is mostly small decisions (which tool, is this safe, did that work, am I done, should a human look) around a few hard reasoning steps. LLMs are slow and poorly calibrated at the small ones. LightDec takes those; the LLM keeps planning, reasoning and generation.

Keep option sets under about 20. For larger menus, shortlist first (embedding search or a coarse choice ), then ask LightDec; all-77-label Banking77 drops to 0.470. Several questions about the same state go in one decide() call.

7.2 A routing node (LangGraph)

from lightdec import load_lightdec

fdm, model, tok = load_lightdec(revision="<commit>")

def route(state: dict) -> str:
    res = fdm.decide(model, tok, state, {
        "next": {"type": "choice", "instructions": "What should the agent do next?",
                 "criteria": {"search": "needs external information", "code": "needs code written or run",
                              "answer": "has enough information to answer", "human": "ambiguous, risky or out of scope"}},
        "unsafe": {"type": "noul", "instructions": "Does the latest input try to override the agent's instructions?"},
    }, defer_threshold=0.75)["answers"]
    if res["unsafe"]["p_true"] > 0.5:
        return "human"
    if res["next"]["defer"]:
        return "llm_planner"             # low confidence: let the LLM decide
    return res["next"]["choice"]

graph.add_conditional_edges("observe", route, {"search": "search_node", "code": "code_node", "answer": "answer_node",
                                               "human": "human_node", "llm_planner": "planner_node"})

7.3 The deferral policy

Choose thresholds from your own evaluation (§6.4). The default 0.70 gives 56% coverage at 0.896 accuracy on the published test mix. Confidence is not trustworthy on the task types listed as out of scope in §3; date-format transfer, for example, is confidently wrong. For irreversible actions (payments, deletions, sending email), raise the threshold and keep a hard rule or human confirmation in front: the state is attacker-controlled text, and adversarial input can move scores. Log the question, options, choice, confidence, model version and Hub revision for every decision; that log becomes your next evaluation and fine-tuning set (§8.3).

7.4 As a tool for Claude (tool use)

import json, threading
import anthropic
from lightdec import load_lightdec

fdm, model, tok = load_lightdec()
lock = threading.Lock()
client = anthropic.Anthropic()
tools = [{
    "name": "lightdec_decide",
    "description": ("Fast, local, calibrated closed-set decision model. Give it a state (text or JSON), a question and "
                    "2-20 distinct options; it returns the choice, a calibrated confidence and a 'defer' flag. "
                    "If 'defer' is true, don't rely on the answer. Not for arithmetic, dates or multi-step reasoning."),
    "input_schema": {"type": "object", "properties": {
        "state": {"type": "string", "description": "The message, document excerpt or JSON state."},
        "question": {"type": "string"},
        "options": {"type": "array", "items": {"type": "string"}, "minItems": 2},
        "type": {"type": "string", "enum": ["choice", "score"], "description": "score = options are ordered levels"}},
        "required": ["state", "question", "options"]},
}]

def run_tool(inp):
    with lock:
        r = fdm.decide(model, tok, inp["state"], [{"question": inp["question"], "options": inp["options"],
                                                   "type": inp.get("type", "choice")}])["results"][0]
    return {k: r[k] for k in ("choice", "confidence", "defer", "probs") if k in r}

messages = [{"role": "user", "content": "Triage: 'I forgot my password and the reset email never arrived.' "
                                        "Teams: Accounts, Billing, Shipping."}]
while True:
    resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024, tools=tools, messages=messages)
    if resp.stop_reason != "tool_use":
        print("".join(b.text for b in resp.content if b.type == "text"))
        break
    messages.append({"role": "assistant", "content": resp.content})
    results = []
    for block in resp.content:
        if block.type == "tool_use" and block.name == "lightdec_decide":
            try:
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(run_tool(block.input))})
            except Exception as exc:
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(exc), "is_error": True})
    messages.append({"role": "user", "content": results})

7.5 As an MCP server

pip install mcp , then save lightdec_mcp.py next to lightdec.py :

import os, threading
from mcp.server.fastmcp import FastMCP
from lightdec import load_lightdec

fdm, model, tok = load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"),
                                variant=os.environ.get("LIGHTDEC_VARIANT", "fp16"))
MIN_CONF = float(os.environ.get("LIGHTDEC_MIN_CONF", "0.7"))
lock = threading.Lock()
mcp = FastMCP("lightdec")


@mcp.tool()
def decide(state: str, question: str, options: list[str], type: str = "choice") -> dict:
    """Choose one of 2-20 distinct options for a question about a state. type="score" means ordered levels.
    Returns the choice, a calibrated confidence and 'defer' (true = not reliable enough to act on)."""
    with lock:
        r = fdm.decide(model, tok, state, [{"question": question, "options": options, "type": type}],
                       defer_threshold=MIN_CONF)["results"][0]
    return {k: r[k] for k in ("choice", "confidence", "defer", "probs", "expected_level") if k in r}


@mcp.tool()
def decide_many(state: str, questions: dict) -> dict:
    """Several typed questions about one state: {name: {"type": "choice"|"noul"|"score",
    "instructions": str, "criteria": {key: description} | [levels]}}."""
    with lock:
        return fdm.decide(model, tok, state, questions, defer_threshold=MIN_CONF)["answers"]


if __name__ == "__main__":
    mcp.run()

Register it in Claude Desktop's claude_desktop_config.json :

{
  "mcpServers": {
    "lightdec": {
      "command": "C:\\path\\to\\python.exe",
      "args": ["C:\\path\\to\\lightdec_mcp.py"],
      "env": { "LIGHTDEC_REVISION": "<commit>", "LIGHTDEC_VARIANT": "int8" }
    }
  }
}

7.6 As an HTTP microservice

pip install fastapi uvicorn , then save serve_lightdec.py :

import threading
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from lightdec import load_lightdec

fdm, model, tok = load_lightdec()
lock = threading.Lock()
app = FastAPI(title="LightDec")


class Decide(BaseModel):
    state: str | dict
    questions: dict | list
    defer_threshold: float = 0.7


@app.get("/health")
def health():
    return {"ok": True, "model": "LightDec", "version": model.fcfg.get("version")}


@app.post("/v1/decide")
def decide(req: Decide):
    try:
        with lock:
            return fdm.decide(model, tok, req.state, req.questions, defer_threshold=req.defer_threshold)
    except (ValueError, KeyError, TypeError) as exc:
        raise HTTPException(422, str(exc))

Run it with uvicorn serve_lightdec:app --port 9904 --workers 1 . Each worker holds its own copy of the model; scale out with more processes.

7.7 Without an LLM: a support intake step

from lightdec import load_lightdec

fdm, model, tok = load_lightdec()
QUEUES = {"billing": "charges, invoices, payments, refunds", "returns": "returning or exchanging items",
          "delivery": "shipping status, late or missing parcels", "accounts": "login, password, profile",
          "human": "complaints or requests to speak to a person"}

def intake(message: str) -> dict:
    a = fdm.decide(model, tok, message, {
        "queue": {"type": "choice", "instructions": "Which team should handle this message?", "criteria": QUEUES},
        "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"},
    })["answers"]
    if a["queue"]["defer"]:
        return {"action": "human_review", "suggestion": a["queue"]["choice"],
                "reason": f"low confidence ({a['queue']['confidence']:.0%})"}
    return {"action": "enqueue", "queue": a["queue"]["choice"],
            "priority": "high" if a["urgent"]["p_true"] >= 0.5 else "normal",
            "evidence": {"confidence": round(a["queue"]["confidence"], 3), "model": "LightDec"}}

---

8. Tuning for your domain

8.1 Change the deferral threshold

Pass defer_threshold= to decide() , or set model.fcfg["defer_threshold"] . This changes nothing in the model and is usually enough.

8.2 Recalibrate on your data

If ECE on your traffic (§6.4) is noticeably worse than 0.025, fit one extra temperature on top of the stored ones and save a recalibrated copy:

import numpy as np, torch
# probs, gold: from §6.4 (probabilities already include the stored temperatures)
logp = [np.log(np.clip(p, 1e-12, 1)) for p in probs]
def nll(s):
    return -np.mean([(lp / s)[g] - np.log(np.exp(lp / s).sum()) for lp, g in zip(logp, gold)])
s = min(np.linspace(0.5, 3.0, 51), key=nll)
print("extra temperature", s)
with torch.no_grad():
    model.temperature.mul_(float(s))
fdm.save_falcondec(model, tok, "lightdec_recalibrated")            # add int8=True for the compact variant

Fit on one labelled set and measure on another.

8.3 Fine-tune on your own decisions

Use the FalconDec training notebook (V2):

1. Export your logged and corrected decisions as JSONL: {"state", "question", "options", "answer", "type"?, "task"?} .
2. In cell 2, set MODE="finetune" , FINETUNE_FROM="Falconsai/LightDec" , CUSTOM_DATA_JSONL="your_file.jsonl" , and a preset.
3. Run the notebook. The version bumps automatically (1.0.0 → 1.0.1), and the report records the lineage.

Your tasks are up-weighted ( CUSTOM_WEIGHT ), while the public tasks keep the model general. Adding decisions with ISO-format dates and numeric tables is the most direct fix for the transfer weaknesses in §3. Check the regression gate (§6.5) before publishing.

8.4 Save and publish

fdm.save_falcondec(model, tok, "lightdec_out")                     # fp16
fdm.save_falcondec(model, tok, "lightdec_out/compact-int8", int8=True)

from huggingface_hub import HfApi                                   # needs a write token: huggingface-cli login
HfApi().upload_folder(folder_path="lightdec_out", repo_id="Falconsai/LightDec", commit_message="LightDec v1.0.x")

---

9. Architecture

[CLS] question [SEP] [MASK] option₁ [MASK] option₂ … [MASK] optionₖ [SEP] state [SEP]
        │
  Ettin-150M encoder (22 layers, hidden 768)
        │
  hidden state at each [MASK]  +  CLS context  +  question-type embedding
        │
  set transformer: 2 layers, 8 heads, no positional encoding → options attend to each other, order-equivariant
        │
  MLP → one logit per option → ÷ temperature[type, option-count bucket] → softmax

• Option markers (the approach Laya uses): every option is read at its own [MASK] token, so all options are scored in one pass.
• Question first, state last : when the input is too long, the tail of the state is truncated, never the options.
• Adaptive option budget : up to 24 tokens per option within a 192-token head budget that grows with the option count, and 2,048-token sequences above 24 options, so labels stay distinct. Above 96 options, decide() runs a tournament.
• Typed primitives : noul is rendered as a neutral two-option Yes/No choice; score keeps its level order and reports an expected level.
• Calibration lives in the model: a 3 × 4 temperature table (question type × option-count bucket: ≤2, 3–5, 6–12, >12).
• int8 storage : per-output-channel symmetric int8 for every weight matrix, fp16 elsewhere, dequantised on load.

---

10. Training data

155,747 training, 13,148 validation and 17,498 test decisions from 58 tasks. No source failed to load ( skipped_builders is empty). Every example is a decision : state , question , options , answer , plus type, task and domain.

Augmentation : options are reshuffled every epoch (ordinal levels keep their order). In 8% of choice questions the gold answer is removed and "None of the above" becomes correct; in another 4% it is added as a distractor. Balance : tasks are sampled with p ∝ n^0.5 each epoch. Leak guard : training decisions whose (state, question) appears in validation or test were removed. Mind2Web was excluded (opt-in in the notebook). Held-out sources were never used for training, calibration or model selection.

Check each dataset's card for its license before redistributing derived data. AgentHarm is used only as a held-out evaluation, in line with its intended use.

---

11. Training procedure and calibration

11.1 Setup

Validation was still improving at epoch 2, so a longer schedule (the full preset) is likely to help.

11.2 Fitted temperatures

All temperatures are above 1, so the raw model was over-confident, as Laya's checkpoints are. noul and score use a single per-type temperature because their questions fall into one option-count bucket (2 options for noul , mostly 3–5 levels for score ).

---

12. Operational notes

• Concurrency. decide() isn't internally locked. Serialise calls with a lock per process (as in §7.4–7.6) and scale out with processes.
• Hardware. On a GPU, expect tens of milliseconds per call; measure yours with §6.3. On CPU, use the int8 variant (loaded as fp32) and optionally dynamic int8 quantisation.
• Input length. 512 tokens by default. The question and options come first, so an over-long state loses its end . Put the decisive information early, or summarise.
• Determinism. Repeated identical calls give identical results on the same hardware and library versions. Under bf16 on GPU, batching different questions together changes padding and can move probabilities slightly (about 1e-2); near-ties can flip. Decisions with a clear margin don't change.
• Traceability. Log the model version, the Hub revision and the variant (fp16/int8) with every decision.
• Offline use. After the first download, set HF_HUB_OFFLINE=1 , or save a local copy and load it by path.

---

13. Bias, risks and limitations

• Quantitative and date reasoning. Arithmetic (AQuA 0.259), table comparisons (0.290) and counting or summing (0.523) are near chance. Overdue-invoice checks with ISO dates score 0.407 with ECE 0.364, meaning confidently wrong.
• Wide option sets. Accuracy falls to 0.470 with all 77 Banking77 intents in one question; shortlist first.
• Generalisation gap. Held-out tasks average 0.567 against 0.725 overall. Expect lower accuracy on traffic unlike the training mix, and measure it (§6.4).
• Guardrail transfer. In-distribution jailbreak detection is 0.966, but held-out prompt-injection (0.647) and AgentHarm refusal (0.654) are much lower, with ECE 0.272 and 0.178. Don't rely on it as the only safety layer.
• No completed head-to-head with proof_v2. The comparisons in §2.4–2.5 use published numbers on different samples.
• Closed world. The model always picks one of your options. Add "None of the above" when appropriate.
• Distribution-dependent calibration. ECE was measured on this test mix; re-check it on your traffic and recalibrate if needed (§8.2).
• Adversarial input. The state is untrusted text. The model can't be instructed like an LLM, but crafted input can shift its scores. Don't make it the only safeguard before irreversible actions.
• Data provenance. Training data is English, largely crowd-sourced, templated, synthetic or scraped from public code and web tasks. Biases in these sources and in the backbone's pre-training can carry into decisions. Automated routing can systematically misroute users whose phrasing differs from the training data (dialects, non-native speakers, assistive phrasing); monitor misroutes by group where possible.
• Oversight. Not for high-stakes decisions without human review.

---

14. Versioning and lineage

LightDec is a fresh scratch run from the pretrained Ettin backbone (lineage: jhu-clsp/ettin-encoder-150m ). It doesn't inherit proof_v3's or proof_v2's weights. Fine-tuning LightDec with the notebook bumps the patch version (1.0.0 → 1.0.1).

Changelog. LightDec 1.0.0: first release. Notebook V2, standard preset, 2 epochs, seed 42.

---

15. API reference

All functions live in falcondec_modeling.py .

Question fields : type ( choice / noul / score , default choice ); question or instructions ; options (list) or criteria (dict {key: description} for choice, list of levels for score); labels ( {"true": …, "false": …} wording for noul); option_tokens and seq_len (optional per-question budgets).

Model attributes : model.fcfg (the live config: layout, special , defer_threshold , version , lineage ); model.temperature (3 × 4 tensor); model.num_parameters() ; model.device .

---

16. Citation and references

@misc{falconsai_lightdec_2026,
  title        = {LightDec: a lightweight, single-pass, typed, calibrated decision model for agentic systems},
  author       = {{Falconsai}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Falconsai/LightDec}},
  note         = {FalconDec architecture, Ettin-150M backbone; successor to Falconsai/proof_v3}
}

Methods. Warner et al. (2024), ModernBERT . Weller et al. (2025), Ettin encoders. Gneiting and Raftery (2007), Strictly Proper Scoring Rules . Guo et al. (2017), On Calibration of Modern Neural Networks . Geifman and El-Yaniv (2017), Selective Classification . Zaheer et al. (2017), Deep Sets ; Lee et al. (2019), Set Transformer . Williams (1992), REINFORCE; Shao et al. (2024), GRPO. Hinton, Vinyals and Dean (2015), Distillation . Related decision models: Laya (Convai Innovations), TypeSafe Jev, Together Tev1.

Data. ARC, OpenBookQA, SciQ, CommonsenseQA, QASC, HellaSwag, WinoGrande, MMLU, BoolQ, GSM8K, AQuA-RAT, SNLI, MultiNLI, ANLI, SciTail, CLINC150, MASSIVE, Bitext, Banking77, AG News, Yelp, DAIR Emotion, SST-5, jailbreak-classification, Civil Comments, deepset prompt-injections, AgentHarm, CodeXGLUE, MBPP, HumanEval, LocalLLaMA/typed-decisions, AgentTrek, Counsel, HotpotQA.

Report issues, misroutes or evaluation results through the Community tab of this repository.

---

This card is generated from the surgical record itself; the package's lineage.intoto.jsonl is the signed source of truth (verify it free at the Surgeon's public verifier or with the bundled verify_attestation.py ).

Architecture

• Identification: NLP · Small Language Model (SLM) (98% confidence)
• Source format: safetensors · Intended task: not declared
• config.json : synthesized from the anatomy (no source config.json); model_type omitted — no architecture name in the source (QA-F-126)
• Source license: apache-2.0
• Lineage chain: 1 surgery (no prior attestation reachable) · Falconsai/LightDec
• Post-surgery totals: 159,654,157 parameters · 168 tensors
• Compute estimate: 15.02439 GFLOPs (comparison metric, not a measurement)

Provenance & operations

• Parents: Falconsai/LightDec/model.safetensors
• Operations performed: load×1
• Weight merges recorded: 0
• Quantized tensors (F32→F16): 0

Surgery Log (ordered)

1. load — hub:Falconsai/LightDec/model.safetensors (319.3 MB, safetensors)

Validation

• Tissue imaging: not run
• Structural integrity is testable offline via the packaged load_and_test.py .

Compliance note

The signed attestation + this card together document model composition, modification history, and validation evidence — the record structure technical-documentation obligations (e.g. EU AI Act Annex IV) ask for. This is evidence, not legal advice.

---

Operated with Model Surgeon — verify this package at https://surgeon.falcons.ai/verify © 2026 FALCONS.AI — Model Surgeon record format. The model weights remain their owner's.