AtomicChat / d1-3B-GGUF

How to Run d1-3B Locally

Built from Liquid AI's published weights with our own importance matrix, and measured on decisions, not just text.

d1-3B is Liquid AI's decision model. It takes a state and answers typed questions in one forward pass: yes or no, a pick from named options, or a score. The answer is read from the model's distribution over the options' tokens, so nothing is generated.

Pick a file

Every number below is measured on one machine. The raw results and logs are in the metrics repo .

• same answer : across 1,285 decisions, how often the file gives the same answer as the BF16 weights published in LiquidAI/d1-3B . The questions are yes/no, choice and score questions over held-out texts in 30 languages and source code. The number of changed answers is in brackets.
• option drift : the mean total variation distance between the file's option probabilities and the original's. 0 means identical.
• KLD and top-1 : full-vocabulary agreement with the original on held-out text.

AD- means Atomic Dynamic: the type is chosen per tensor instead of taken from a llama.cpp preset.

• The token table, which d1-3B also uses as its output head, never goes below Q6_K.
• Attention stays at Q8_0 or Q6_K.
• ffn_down in the first and last four blocks takes one step more than the middle.

The projector for images comes as mmproj-d1-3B-BF16 and mmproj-d1-3B-Q8_0 . We did not measure image decisions.

How these compare to Liquid's own GGUFs

• 6 October: Liquid published GGUFs of d1-3B at LiquidAI/d1-3B-GGUF .
• 7 October: Liquid updated the model's weights and the benchmark numbers in its card ("New weights for the same model", PR #4 in LiquidAI/d1-3B ). Later that day the GGUF repo was taken down.

We measured those GGUFs on 7 October, before they were removed, with the same harness and against the same BF16 reference.

• Those GGUFs were built from the previous version of the weights. Every linear layer differs from the current model.safetensors by about 2-3% relative, while the embeddings and norms match exactly, as you would expect from two fine-tunes of the same base.
  • We ran the current checkpoint with Liquid's own PyTorch code on 300 decisions.
  • Our BF16 file matches it with a mean option KL of 0.0002. Liquid's BF16 file is at 0.0078.
  • Our files are converted from the current weights, the ones Liquid's benchmark tables now refer to.
  • Most of the gap between the two 4-bit files above comes from that older starting point, not from quantization (see below).

Quantization alone. To separate the two effects, each file is measured against its own BF16 base: ours against the current weights, and Liquid's against its own BF16 GGUF of the previous weights.

• The two 4-bit files change the same number of answers relative to their own base.
• Ours moves the option probabilities less: about half the KL and a quarter less drift, at a slightly smaller size. For a model whose answers are calibrated probabilities, that is the part that counts.
• Those files also did not load in stock llama.cpp. They carried lfm2.decision.type = d1 , which llama.cpp did not know (commit 18b5f8b ): unsupported decision model type: d1 .
• Upstream support is on its way. Liquid's own PR, ggml-org/llama.cpp#30110 , adds d1-3B under the type lfm2-d1 . Our files leave that key out and load today as a plain LFM2 model. Once the PR is merged we will add the key and its systemone template, so that /v1/systemone works with them.

Running it

llama-server -m d1-3B-AD-Q4_K_M.gguf -ngl 99 -c 8192

llama.cpp's /v1/systemone endpoint does not support d1 yet ( #30110 adds it). Until then, the answer is a few lines of Python:

• render the prompt with Liquid's own prompt.py ;
• ask the server for one token with its top log-probabilities;
• take a softmax over the options' tokens.

import json, math, sys, urllib.request
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

src = snapshot_download("LiquidAI/d1-3B", allow_patterns=["prompt.py", "tokenizer*"])
sys.path.insert(0, src)
import prompt

tok = AutoTokenizer.from_pretrained(src)
q = prompt.as_question({"type": "choice", "instructions": "Which team should handle this?",
                        "criteria": {"billing": "Charges, refunds, invoices", "technical": "App or site faults",
                                     "fraud": "Suspected unauthorised use"}})
ids = tok(prompt.render(tok, "I was charged twice this month, please refund one of them.", q, tok.bos_token),
          add_special_tokens=False)["input_ids"]
req = urllib.request.Request("http://127.0.0.1:8080/completion", json.dumps(
    {"prompt": ids, "n_predict": 1, "n_probs": 512, "post_sampling_probs": False}).encode(),
    {"Content-Type": "application/json"})
top = json.load(urllib.request.urlopen(req))["completion_probabilities"][0]["top_logprobs"]
lp = {t["id"]: t["logprob"] for t in top}
s = [max(lp.get(i, min(lp.values())) for i in g) for g in prompt.readout_ids(tok, q)]
p = [math.exp(x - max(s)) for x in s]
print({name: round(v / sum(p), 3) for name, v in zip(q.criteria, p)})
# {'billing': 0.985, 'technical': 0.01, 'fraud': 0.005}

noul (yes or no) and score questions work the same way. See the original card for the question schema.

How we made and measured them

• BF16: converted from LiquidAI/d1-3B (revision da1fe36 ) with llama.cpp 18b5f8b . It matches Liquid's PyTorch code on 300 decisions with a mean option KL of 0.0002. The 9 changed answers there are near-ties, with a largest option difference of 0.035. The projector is tensor-for-tensor identical to Liquid's.
• Importance matrix: 1.5M tokens of decision prompts, 3,206 in all. The states come from the calibration corpora pool (Wikipedia in 30 languages, code, structured files). Each carries yes/no, choice and score questions and is rendered with the model's own prompt code.
• Decisions: 1,285 questions over the held-out calib-corpora eval/neutral and eval/code texts, none of them in the calibration set. They are answered with the readout above. The scripts are in the metrics repo.
• Text: llama-perplexity --kl-divergence over eval/neutral , 93 chunks of 4,096 tokens.
• Setup: an NVIDIA H100 and llama.cpp 18b5f8b for every file.

Model details

• Base: LiquidAI/d1-3B , a decision model post-trained from LFM2.5-VL-3B: 3.1B parameters, 32K context, a 128K vocabulary and a SigLIP2 vision encoder.
• License: LFM Open License v1.0; see LICENSE .