On-device build of
Nanbeige/Nanbeige4.2-3B
for Google's
LiteRT-LM
runtime — an Apache-2.0 3B-class
looped-transformer
reasoning model (its 22 layers run twice per token,
num_loops=2
).
Note on speed: the loop makes this model
~2× the per-token compute of a normal 3B
(that's where its quality comes from) — expect roughly 8B-class decode rates on CPU.
The loop is unrolled at export: 44 layer executions over 22 shared-weight layers, one KV-cache slot per (loop, layer) — 44 slots. Weights are stored once; only compute and KV double.
The chat template auto-opens the model's
<think>
block (this is how the official template works — the model starts already inside its reasoning).
Quality (GSM8K, n=50, 0-shot CoT, official sampling)
build
GSM8K
bf16 (transformers 4.51, reference)
94.0% (47/50)
this int4 .litertlm (LiteRT-LM CPU)
90.0% (45/50)
Local gate: 8/8 basic-quality questions, clean
<think>…</think>
closure, ChatML stop.
⚠ Usage notes
Sampling is required.
The model collapses under greedy decoding (official recipe:
temperature 0.6, top_k 20, top_p 0.95
— see its
generation_config.json
). Configure the runtime session accordingly; do not run at temperature 0.
CPU backend.
The current GPU delegate produces incorrect output on this 44-layer unrolled graph; run on CPU.
Reasoning outputs open inside
<think>
; the visible answer follows
</think>
.
Run it
Android: push the file and import it in
Google AI Edge Gallery
(
+
→ import, keep CPU).
CLI / other platforms: see the LiteRT-LM repo for
litert-lm
runtime usage.
The conversion script (
scripts/convert_nanbeige42.py
) documents the two traps this model hits: transformers-5.x zeroes the modeling's init-computed rotary
inv_freq
buffer (rope must be recomputed from config), and the looped architecture needs a 44-slot KV cache registered through the export pipeline's
cache_implementation
hook.
Runs of litert-community Nanbeige4.2-3B on huggingface.co
357
Total runs
0
24-hour runs
6
3-day runs
122
7-day runs
357
30-day runs
More Information About Nanbeige4.2-3B huggingface.co Model
Nanbeige4.2-3B huggingface.co is an AI model on huggingface.co that provides Nanbeige4.2-3B's model effect (), which can be used instantly with this litert-community Nanbeige4.2-3B model. huggingface.co supports a free trial of the Nanbeige4.2-3B model, and also provides paid use of the Nanbeige4.2-3B. Support call Nanbeige4.2-3B model through api, including Node.js, Python, http.
Nanbeige4.2-3B huggingface.co is an online trial and call api platform, which integrates Nanbeige4.2-3B's modeling effects, including api services, and provides a free online trial of Nanbeige4.2-3B, you can try Nanbeige4.2-3B online for free by clicking the link below.
litert-community Nanbeige4.2-3B online free url in huggingface.co:
Nanbeige4.2-3B is an open source model from GitHub that offers a free installation service, and any user can find Nanbeige4.2-3B on GitHub to install. At the same time, huggingface.co provides the effect of Nanbeige4.2-3B install, users can directly use Nanbeige4.2-3B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.