litert-community / Nanbeige4.2-3B

huggingface.co
Total runs: 357
24-hour runs: 0
7-day runs: 122
30-day runs: 357
Model's Last Updated: August 10 2026
text-generation

Introduction of Nanbeige4.2-3B

Model Details of Nanbeige4.2-3B

Nanbeige4.2-3B — LiteRT-LM (.litertlm)

On-device build of Nanbeige/Nanbeige4.2-3B for Google's LiteRT-LM runtime — an Apache-2.0 3B-class looped-transformer reasoning model (its 22 layers run twice per token, num_loops=2 ).

File model.litertlm (~2.4 GB)
Quantization int4 weights — blockwise (block 32) + OCTAV optimal-clipping, symmetric; embedding INT8
Context (KV cache) 4096 (44 KV slots: 22 layers × 2 loops)
Base model Nanbeige/Nanbeige4.2-3B (Apache-2.0)
Decode speed ~3.2 tok/s (iPhone 17 Pro, CPU; loads ~4.4 s, ~1.8–2.1 GB peak) · ~10 tok/s (Mac M-series, CPU engine)

Note on speed: the loop makes this model ~2× the per-token compute of a normal 3B (that's where its quality comes from) — expect roughly 8B-class decode rates on CPU.

What's in the box
  • model.litertlm — int4 (blockwise-32, OCTAV clipping) + int8 embedding, 4096-token KV cache, ~2.4 GB.
  • The loop is unrolled at export: 44 layer executions over 22 shared-weight layers, one KV-cache slot per (loop, layer) — 44 slots. Weights are stored once; only compute and KV double.
  • The chat template auto-opens the model's <think> block (this is how the official template works — the model starts already inside its reasoning).
Quality (GSM8K, n=50, 0-shot CoT, official sampling)
build GSM8K
bf16 (transformers 4.51, reference) 94.0% (47/50)
this int4 .litertlm (LiteRT-LM CPU) 90.0% (45/50)

Local gate: 8/8 basic-quality questions, clean <think>…</think> closure, ChatML stop.

⚠ Usage notes
  • Sampling is required. The model collapses under greedy decoding (official recipe: temperature 0.6, top_k 20, top_p 0.95 — see its generation_config.json ). Configure the runtime session accordingly; do not run at temperature 0.
  • CPU backend. The current GPU delegate produces incorrect output on this 44-layer unrolled graph; run on CPU.
  • Reasoning outputs open inside <think> ; the visible answer follows </think> .
Run it

Android: push the file and import it in Google AI Edge Gallery ( + → import, keep CPU). CLI / other platforms: see the LiteRT-LM repo for litert-lm runtime usage.

Reproduce

Converted with hf-to-litertlm :

bash scripts/reproduce_llm.sh nanbeige4.2-3b

The conversion script ( scripts/convert_nanbeige42.py ) documents the two traps this model hits: transformers-5.x zeroes the modeling's init-computed rotary inv_freq buffer (rope must be recomputed from config), and the looped architecture needs a 44-slot KV cache registered through the export pipeline's cache_implementation hook.

Runs of litert-community Nanbeige4.2-3B on huggingface.co

357
Total runs
0
24-hour runs
6
3-day runs
122
7-day runs
357
30-day runs

More Information About Nanbeige4.2-3B huggingface.co Model

More Nanbeige4.2-3B license Visit here:

https://choosealicense.com/licenses/apache-2.0

Nanbeige4.2-3B huggingface.co

Nanbeige4.2-3B huggingface.co is an AI model on huggingface.co that provides Nanbeige4.2-3B's model effect (), which can be used instantly with this litert-community Nanbeige4.2-3B model. huggingface.co supports a free trial of the Nanbeige4.2-3B model, and also provides paid use of the Nanbeige4.2-3B. Support call Nanbeige4.2-3B model through api, including Node.js, Python, http.

litert-community Nanbeige4.2-3B online free

Nanbeige4.2-3B huggingface.co is an online trial and call api platform, which integrates Nanbeige4.2-3B's modeling effects, including api services, and provides a free online trial of Nanbeige4.2-3B, you can try Nanbeige4.2-3B online for free by clicking the link below.

litert-community Nanbeige4.2-3B online free url in huggingface.co:

https://huggingface.co/litert-community/Nanbeige4.2-3B

Nanbeige4.2-3B install

Nanbeige4.2-3B is an open source model from GitHub that offers a free installation service, and any user can find Nanbeige4.2-3B on GitHub to install. At the same time, huggingface.co provides the effect of Nanbeige4.2-3B install, users can directly use Nanbeige4.2-3B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Nanbeige4.2-3B install url in huggingface.co:

https://huggingface.co/litert-community/Nanbeige4.2-3B

Url of Nanbeige4.2-3B

Provider of Nanbeige4.2-3B huggingface.co

litert-community
ORGANIZATIONS

Other API from litert-community