CMSManhattan / JiRackCoderReasoning_32b

JiRack Coder Reasoning 32B INT4

A fast and efficient coding assistant with a clean built-in web UI, powered by Qwen3.0-Coder-32B-Instruct base and optimized using Microsoft ONNX Runtime.

Quick Start

Watch the JiRack Coder 32B in action: DEMO : JiRack Coder Reasoning 32B Web UI

Run with Docker

--Default CPU--

• docker run -d
  --name jirack_coder_reasoning_32b
  -p 7869:7869
  --restart unless-stopped
  cmsmanhattan/jirack_coder_32b_int4_qwenbase:latest

--Multi CPU--

• docker run -d
  --name jirack_coder_reasoning_32b
  -p 7869:7869
  --restart unless-stopped
  --memory=48g
  --cpus=16
  cmsmanhattan/jirack_coder_32b_int4_qwenbase:latest

---GPU-- -- comming soon

• docker run -d
  --name jirack_coder_reasoning_32b
  -p 7869:7869
  --gpus all
  --restart unless-stopped
  cmsmanhattan/jirack_coder_32b_int4_gpu_qwenbase:latest

Access the UI

Once the container is running, open your browser and navigate to:

http://localhost:7869

This opens the JiRack Coder UI — a clean web interface designed for coding.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack Coder UI.

Licensing

• The JiRack Coder 32B model is provided under a commercial enterprise license .
• All JiRack UI clients are provided under a commercial license.
• However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.

JiRack Coder 14B is available under a lighter commercial license (~$12 per user/year).

For commercial licensing, cluster deployment, or enterprise use of the JiRack Coder 32B and JiRack Coder 14B, please contact us.

• JiRack MS Windows 11 Desktop chat client with ollama API setup: https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
• Live email chat with model via [email protected]

Hardware Recommendations for AMD Systems

It is significantly heavier than JiRack Coder 14B INT4

Recommended Hardware for JiRack Coder Reasoning 32B INT4. It is one docker container

Important Memory Notes

Even though the 32B INT4 model itself takes approximately 12–14 GB , we recommend at least 48GB VRAM for the following reasons:

• KV-cache consumption during generation (especially with long context)
• ONNX Runtime overhead and temporary buffers
• System stability and to avoid Out of Memory errors
• Room for larger context windows

Minimum recommended: 48GB VRAM (dual RX 7900 series or MI300X)
Ideal: 48–64GB VRAM

For pure CPU inference (no GPU), we recommend at least 128GB system RAM (Ryzen 9 7950X/9950X or better).

I will use the default model in full FP32 precision for quantization, allowing us to find the optimal balance between model size and performance.

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

• Email: [email protected]
• Phone: +1 (516) 777-0945
• Location: New York, USA