CMSManhattan / JiRack_10b

JiRack 10B INT8

A fast and efficient coding assistant with a clean, modern built-in web UI. Powered by Meta Llama 3.1 8B Instruct weights and a fully refactored architecture optimized for a 10B-scale model. The model was specifically designed for high-performance tuning with advanced ternary quantization options. The next version will release JiRack Ternary 10B — a highly optimized ternary model delivering exceptional speed and efficiency using Microsoft ONNX Runtime.

JiRack android client DEMO: https://www.youtube.com/watch?v=SaO6Jfb8R68

CMS Manhattan RAG & Email reply & Document and Emails Analytics https://www.youtube.com/watch?v=KRu2nLEh_6g&t=78s

So I do not read my emails I am asking my JiRack to tell me news! Welcom to buy CMS Manhattan AI front office solution

Traning the model

It is easy to train on Blackwell 96 Gb VRAM. So you do not need data center for tune-time or QLoRa on cheap GPU card . Let me know if you need code !

Quick Start

Watch the JiRack 10B in action: Run on docker it.

Run with Docker

---

--Default CPU--

• docker run -d
  --name jirack_10b
  -p 7869:7869
  --restart unless-stopped
  cmsmanhattan/jirack_10b_int8:latest

--Multi CPU--

• docker run -d
  --name jirack_10b
  -p 7869:7869
  --restart unless-stopped
  --memory=20g
  --cpus=12
  cmsmanhattan/jirack_10b_int8:latest

---GPU--

• docker run -d
  --name jirack_10b
  -p 7869:7869
  --gpus all
  --restart unless-stopped
  cmsmanhattan/jirack_10b_gpu_int8:latest

---

Access the UI

Once the container is running, open your browser and navigate to:

http://localhost:7869

This opens the JiRack UI — a clean web interface designed for chat.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack Chat UI.

Licensing

• The JiRack 10B model is provided under a commercial enterprise license .
• All JiRack UI clients are provided under a commercial license.
• However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.

Subscription Plans

Ready to Deploy JiRack?

Get immediate access to the repositories, architecture blueprints, and deployment containers.

3. JiRack Enterprise price:

-- It is about 36$ per user for year .

3. JiRack private price:

-- It is about 12$ per user for year .

For commercial licensing, cluster deployment , performance tuning , or enterprise use of the JiRack 10B, please contact us.

• JiRack android chat client with voice and ollama API setup : https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/app-release.apk or ( Google play)
• JiRack MS Windows 11 Desktop chat client with ollama API setup: https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
• Live email chat with model via [email protected]

Hardware Recommendations for AMD Systems

Recommended Hardware for JiRack Coder10B INT8 . It is one dcoker container

Important Memory Notes

Even though the 10B INT8 model itself takes approximately 8–9 GB , we recommend at least 24GB VRAM for the following reasons:

• KV-cache consumption during generation (especially with long context)
• ONNX Runtime overhead and temporary buffers
• System stability and to avoid Out of Memory errors
• Room for larger context windows

Minimum recommended: 24GB VRAM (RX 7900 series)
Ideal: 24–32GB VRAM

For pure CPU inference (no GPU), we recommend at least 64GB system RAM (Ryzen 9 7950X/9950X).

---

I added the default model in full FP32 precision, which is approximately 62 GB in size. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.

📧 Contact & Licensing

For joint venture opportunities, hardware integration, or licensing inquiries:

• Email: [email protected]
• Phone: +1 (516) 777-0945
• Location: New York, USA