CMSManhattan / JiRackDeltaNet_27b

Qwen 3.8 27b migrated to Ternary Architedure

• Benefits high quality CPU inference TQ_2 on Llama.cpp and Ollama via QAT
• Robotcs, Routing, Coding, Multimedia, Advanced tool calling via JiRackDeltaNetTokenizer
• JiRack DeltaNet understand video and images that best for Robotics also

JiRack DeltaNet 27B (CPU)

A fast and efficient 27B model optimized for CPU inference. Built on a Qwen3.8-style DeltaNet architecture (hybrid attention + SSM), with an updated tokenizer that includes Routing , Media , Vision , Sound , Tool call , and Robotics tags. Ready-to-run GGUF quantizations, and native Ollama support with reasoning disabled by default for fast, direct responses.

• JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.
• Subscription: $1 per month per user (updated license for non-company use).
• Corp Subscription: $3 per month per user (updated license for company use).
• It works without subscription but send message about subscription

JiRack service options

• Current quantizations were done from the FP16 model.
• If you need custom compression or fine-tuning, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.
• Plus double QAT via ONNX QAT.
• Adapt train process to avoid catastrophic forgetting with NDA
• Adapt train process to avoid fast plateau in training with NDA
• Adapts to agentic or instruct models for tool calling, using the JiRack tokenizer to enable high-quality tool calling on small models — built as a domain-specific tool expert.
• Deployment and scale

Ollama production support

• JiRack DeltaNet 27B runs natively on Ollama with reasoning disabled by default (no forced <think> blocks).
• Available now: cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning — https://ollama.com/cmsmanhattan
• Runtime override also supported: ollama run cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning --think=false
• Follow fresh Ollama platform updates

Spring Boot AI tool calls examples for JiRack DeltaNet series

• Tool call library on java for Enterprise https://github.com/alibaba/spring-ai-alibaba

GoEx AI tool calls examples for JiRack DeltaNet series

• Tool call library on python https://github.com/ShishirPatil/gorilla

JiRack DeltaNet tool calls to boost tool call quality

• Use JiRack Precision tokenizer tags for tool calls with ToolBench https://github.com/OpenBMB/ToolBench
• https://huggingface.co/xalss/Qwen2-7B-Instruct-glaive-function-calling
• https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1
• Add JiRack tool call tags in the dataset and modify tool call processor if needed

JiRack RoboTech

• Advanced Tokenizer with Robotics & Routing & Tool calls Tokenizer and other
• CMSManhattan/JiRackDeltaNetTokenizer

Available Variants on Docker with UI

Quick Start

Run with Ollama (recommended — reasoning off by default)

• https://ollama.com/cmsmanhattan

ollama pull cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning
ollama run cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning "What is the capital of France?"

Run with Docker

• 27B docker images can be provided by request.
• Build docker on local from source or request from me.
• Docker images use hidden reasoning logic to make chat clear

Default CPU (Q4 recommended)

docker run -d \
  --name jirack_deltanet_27b \
  -p 7869:7869 \
  --restart unless-stopped \
  cmsmanhattan/jirack_deltanet_27b-cpu-q4:latest

Q3

docker run -d \
  --name jirack_deltanet_27b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack_deltanet_27b-cpu-q3:latest

Q2 (lowest memory)

docker run -d \
  --name jirack_deltanet_27b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack_deltanet_27b-cpu-q2:latest

Full precision

docker run -d \
  --name jirack_deltanet_27b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack-deltanet-27b-cpu:latest

Multi CPU

docker run -d \
  --name jirack_deltanet_27b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  --memory=32g \
  --cpus=8 \
  cmsmanhattan/jirack-deltanet-27b-q4:latest

Docker Compose Example

services:
  jirack:
    image: cmsmanhattan/jirack_deltanet_27b-cpu-q4:latest
    container_name: jirack_deltanet_27b
    ports:
      - "7869:7869"
    volumes:
      - .:/app
      - ./web:/app/web
    environment:
      - MAX_TOKENS=2048
      - TEMPERATURE=0.7
      - TOP_P=0.8
      - TOP_K=20
      - DEFAULT_STREAM=False
      - INTRA_THREADS=4
      - USE_ENV_ALLOCATOR=1
      - THREADS=16
      - THREADS_BATCH=16
    deploy:
      resources:
        limits:
          memory: 32g

Ollama platform

• CPU without JiRack UI
• GPU without JiRack UI
• Ollama images displays reasoning logic by default but it can be off .
• Use JiRack UI from https://www.jirack.com
• cmsmanhattan/JiRackDeltaNet_27b-reasoning:latest 54 GB
• cmsmanhattan/JiRackDeltaNet_27b-q2-reasoning:latest 10 GB
• cmsmanhattan/JiRackDeltaNet_27b-q3-reasoning:latest 13 GB
• cmsmanhattan/JiRackDeltaNet_27b-q4-reasoning:latest 16 GB
• cmsmanhattan/JiRackDeltaNet_27b-q6-reasoning:latest 22 GB
• cmsmanhattan/JiRackDeltaNet_27b-q8-reasoning:latest 29 GB

Access the UI

Once the container is running, open your browser and navigate to: http://localhost:7869 This opens the JiRack UI — a clean web interface.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack UI.

Licensing

• The JiRack DeltaNet 27B model is provided under a commercial license ($12 per user per year).
• All JiRack UI clients are provided under a commercial license.
• However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.

For commercial licensing, cluster deployment, or enterprise use of JiRack models, please contact us.

• JiRack MS Windows 11 Desktop Client (with Ollama API): https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
• Live email chat with the model: [email protected]

Hardware Recommendations

Recommended Hardware for JiRack DeltaNet 27B (single Docker container)

Important Memory Notes

Even though the quantized 27B models are relatively compact for their size, we recommend the following for best experience:

• Q4_K_M: 18–24 GB system RAM minimum
• Q3_K_M / Q2_K: 15–20 GB system RAM
• Full precision: 48 GB+ system RAM recommended

Reasons for extra headroom:

• KV-cache consumption during generation
• Runtime overhead and temporary buffers
• System stability and avoiding out-of-memory errors
• Room for larger context windows

Minimum recommended (Q4): 18 GB system RAM Ideal: 24–32 GB system RAM

I added the default model in full precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.

Architecture Notes

• Qwen3.8-style DeltaNet architecture : hybrid attention + SSM design ( qwen35 in GGUF metadata)
• Updated tokenizer : Extended with new special tags for Routing , Tool call , and Robotics
• No-forced-reasoning support : chat template patched so <think> blocks default to closed; native support on Ollama
• Hidden 5120, 65 layers, attention heads 24 / KV heads 4, feed-forward 17408
• SSM: conv kernel 4, state size 128, group count 16, time-step rank 48, inner size 6144
• RoPE θ = 10,000,000, RMSNorm ε = 1e-6
• Context length: up to 262,144 tokens
• Ready-to-run GGUF quantizations (Q2_K, Q3_K_M, Q4_K_M)

Benchmarks

JiRack DeltaNet 27B is built on the Qwen3.5/Qwen3.8-style DeltaNet architecture. The table below reproduces the published base-model benchmark results from Qwen/Qwen3.8-27B for reference — these reflect the upstream base model's capabilities, not JiRack-specific fine-tuning or quantization results.

Text Performance

VL Performance

Source: Qwen/Qwen3.8-27B model card . Best result in each row is bolded. Empty cells (--) indicate results not yet available. See the source card for full evaluation methodology and footnotes.

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

• Email: [email protected]
• Phone: +1 (516) 777-0945
• Location: New York, USA

License

MIT License