CMSManhattan / JiRackDeltaNet_180b

Qwen3.8-Flash-Next 180B migrated to the JiRack Ternary Architecture

• A 180B-parameter Mixture-of-Experts model with only 6B parameters active per token , which makes high-quality CPU inference realistic at this scale
• Migrated to the JiRack ternary architecture (BitLinear, per-tensor absmean ternary weights) with a lambda-warmup QAT pipeline, targeting TQ2_0 CPU inference on llama.cpp and Ollama
• Robotics, Routing, Coding and Advanced tool calling via the JiRack tool-call workflow
• The Qwen3.8-Flash-Next base model understands images and video; the JiRack build is currently text-only (see Current status )

PARTNERSHIP

• NVIDIA
• FISERV

JiRack DeltaNet 180B (CPU)

A 180B MoE model built on the Qwen3.8-Flash-Next architecture: Gated DeltaNet linear attention interleaved with Qwen Sparse Attention (QSA), 512 experts with top-10 routing plus a shared expert, gated residual (hyper-connection) streams, and a 51B-parameter hashed n-gram embedding table. Only ~6B parameters are computed per token, so throughput on CPU is closer to a 6B dense model while quality is that of a much larger one.

• JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.

Current status

Important: until QAT is complete, the JiRack 180B weights are numerically identical to the base model (lambda = 0 means BitLinear is a pure passthrough). The ternary benefits described on this card apply after QAT.

JiRack service options

• If you need custom compression or fine-tuning, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.
• Plus double QAT via ONNX QAT.
• Adapt train process to avoid catastrophic forgetting with NDA
• Adapt train process to avoid fast plateau in training with NDA
• MoE-aware QAT: routers are never quantized and stay frozen by default, with built-in router health monitoring (routing agreement, entropy collapse, expert load) during training
• Adapts to agentic or instruct models for tool calling, using the JiRack tokenizer to enable high-quality tool calling — built as a domain-specific tool expert.
• Deployment and scale

JiRack Coding Agent IDE

• Agent Coding IDE for JiRack models, running via Ollama on a home PC or server
• A good alternative to agent coding IDEs such as Cursor, Windsurf or Devin, and safer: it asks you to review and apply changes.
• Web site https://www.jirack.com
• Plugin https://marketplace.eclipse.org/content/jirack-coding-agent

Ollama support

• 180B Ollama builds are planned after QAT. Ollama needs a llama.cpp version that supports the qwen4exp architecture.
• Qwen3.8-Flash-Next thinks by default ( <think>...</think> ); JiRack builds will ship with reasoning disabled by default, as for JiRack 27B.
• Follow https://ollama.com/cmsmanhattan for releases.

Spring Boot AI tool calls examples for JiRack DeltaNet series

• Tool call library on java for Enterprise https://github.com/alibaba/spring-ai-alibaba

GoEx AI tool calls examples for JiRack DeltaNet series

• Tool call library on python https://github.com/ShishirPatil/gorilla

JiRack DeltaNet tool calls to boost tool call quality

• Use JiRack Precision tokenizer tags for tool calls with ToolBench https://github.com/OpenBMB/ToolBench
• https://huggingface.co/xalss/Qwen2-7B-Instruct-glaive-function-calling
• https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1
• Add JiRack tool call tags in the dataset and modify tool call processor if needed

JiRack RoboTech

• Advanced Tokenizer with Robotics & Routing & Tool calls Tokenizer and other
• CMSManhattan/JiRackDeltaNetTokenizer
• The 180B build currently uses the original Qwen3.8 tokenizer (vocabulary 248,320).

Planned GGUF variants

Not yet published. Sizes are estimates from the parameter count (~180B stored); actual files will differ.

Quick Start

Run the JiRack checkpoint directly (PyTorch, CPU)

The JiRack checkpoint is a directory of safetensors shards that is memory-mapped lazily, so it runs even when the model is larger than RAM (slower: pages are read from disk as experts are used).

python test_jirack_180b_generate.py   # quick generation smoke test
python chat_jirack_180b.py            # interactive chat

Build a GGUF

python export_180b_to_hf_safetensors.py \
    --checkpoint /data/qwen38_180b_checkpoint_safetensors \
    --out_dir /data/qwen38_180b_hf_export
python convert_hf_to_gguf.py /data/qwen38_180b_hf_export \
    --outfile JiRackDeltaNet_180b.gguf --outtype bf16

After QAT, add --bake_ternary to the export to write the ternary weights.

Docker

• 180B Docker images with the JiRack UI will be provided after QAT, or by request.

Recommended sampling parameters

From the base model card:

• Thinking mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
• Instruct (non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Access the UI

Once a JiRack container is running, open your browser at http://localhost:7869 . This opens the JiRack UI — a clean web interface. The port can be changed from the Settings panel.

Licensing

This repository is dual-licensed: the model weights and the JiRack code are licensed separately.

• Weights: the Qwen Community License continues to apply to the weights, including the JiRack-converted and quantized versions. Keep the Qwen copyright and license notice with all copies. Commercial Model-as-a-Service and AI work-assistant (coding / office) products built on these weights require a separate license from Qwen — review the license terms before such use.
• Code: the MIT License covers the JiRack source code only. It does not change the license of the weights.
• The Docker image with UI and pre-built Ollama quantizations are separate commercial products.
• All JiRack UI clients are provided under a commercial license. The UI clients can be used for free together with the official JiRack Docker containers, as long as they are not redistributed separately.

For commercial licensing, cluster deployment, or enterprise use of JiRack models, please contact us.

• JiRack MS Windows 11 Desktop Client (with Ollama API): https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
• Live email chat with the model: [email protected]

Hardware Recommendations

Recommended hardware for JiRack DeltaNet 180B

Important Memory Notes

• The whole model must be resident (or memory-mapped) even though only ~6B parameters are computed per token: the router can pick any of the 512 experts in every layer.
• 36 of the 48 layers use Gated DeltaNet with a fixed-size recurrent state, so long contexts need far less KV cache than a full-attention model of this size.
• Leave headroom for runtime buffers and the attention KV cache of the 12 full-attention layers.

Architecture Notes

• Qwen3.8-Flash-Next architecture ( qwen4_exp in Hugging Face, qwen4exp in llama.cpp)
• Parameters: 125B core + 51B n-gram embedding + 4B MTP (≈180B stored), 6B activated per token
• Layers: 48 = 12 × (3 × Gated DeltaNet → MoE, 1 × Gated Attention with QSA → MoE)
• Hidden size 2560, carried as 4 parallel hyper-connection streams (gated residual replaces the usual pre/post norms)
• MoE: 512 experts (intermediate 640), top-10 routing, plus 1 shared expert with a sigmoid gate
• Gated DeltaNet: 16 key heads × 128, 48 value heads × 128, conv kernel 4, sigmoid output gate
• Gated Attention: 24 query heads, 2 KV heads, head dim 256, partial rotary 0.25, interleaved mRoPE (11/11/10), θ = 10,000,000
• QSA (Qwen Sparse Attention): keys pooled in blocks of 4 tokens; each query attends to the top 2048 tokens' blocks plus the local tail
• PLE n-gram embedding on layer 2: hashed bi- and trigrams, 16 heads, 320M-row table (51B parameters) in 128 shards
• MTP head: 1 layer for speculative decoding
• RMSNorm ε = 1e-6, vocabulary 248,320, context 262,144 tokens natively (extensible to 1M in the base model)

JiRack ternary scheme

• Ternarized (BitLinear): DeltaNet in_proj_qkv / in_proj_z / out_proj , attention q/k/v/o_proj , all routed experts and the shared expert — 74,028 layers
• Always full precision: MoE routers and the shared-expert gate (never quantized, frozen by default during QAT), hyper-connections, all norms, conv1d, A_log / dt_bias , QSA indexer, PLE projections and n-gram table, embeddings, lm_head , MTP head
• Weights: per-tensor absmean ternary {-γ, 0, +γ} ; activations during training: per-token int8
• QAT: continuous lambda warmup from full precision (λ = 0) to fully ternary (λ = 1) with a straight-through estimator

Benchmarks

JiRack DeltaNet 180B is built on Qwen3.8-Flash-Next. The tables below reproduce the published base-model results from Qwen/Qwen3.8-Flash-Next for reference — they reflect the upstream base model's capabilities, not JiRack-specific QAT or quantization results.

Language

Vision-Language (base model)

Source: Qwen/Qwen3.8-Flash-Next model card . Best result in each row is bolded. Empty cells (--) indicate results not available or not applicable. See the source card for evaluation harnesses, settings and footnotes.

Citation

@techreport{qwen2026design,
    title       = {On the Design of {Qwen3.8-Next} Architecture: Evaluation, Efficiency, and Training Stability},
    author      = {{Qwen Team}},
    institution = {Alibaba Group},
    month       = {August},
    year        = {2026}
}

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

• Email: [email protected]
• Phone: +1 (516) 777-0945
• Location: New York, USA

License

• Model weights: Qwen Community License 1.0 — see LICENSE . Copyright 2026 Qwen.
• JiRack code and tools: MIT License — see LICENSE-CODE . © 2025-2026 Konstantin Vladimirovich Grabko, CMS Manhattan JiRack Technology.