CMSManhattan / JiRackNative_3b

JiRack Robotics - TernaryTransformer3B (Pre-training Phase)

JiRack Robotics has officially kicked off the multi-shard pre-training phase for its latest 3B parameter robotics model .

Running under the JiRackTrain pipeline on the enterprise infrastructure cluster ( root@jirack1 ), the training initializes the next-generation TernaryTransformer3B architecture, tightly coupled with the advanced JiRack Pro Tokenizer .

Model Details

• Model Name : TernaryTransformer3B (3 Billion Parameters)
• Architecture : TernaryTransformer (custom ternary bit-response logic)
• Tokenizer : JiRack Pro Tokenizer (128K) — 347 active language editions of Wikipedia + specialized robotic action tokens
• Pre-training Dataset : JiRack-Pretrain-Dataset

Key Metrics (Step ~838/1000 on Shard 7) on Blackwell 96 Gb VRAM

• Iteration Speed : 2.36s/it (network tensor loading)
• Current Loss : 1.2665
• Moving Average Loss : 1.6637
• Perplexity (PPL) : ppl=5.3

[Shard 8/100] jirack_pretrain_chunk_7.pt
Training jirack_pretrain_chunk_7.pt:  83%|█████▊ | 208/250 [08:09<01:38,  2.36s/it, loss=1.2665, avg_loss=1.6637, ppl=5.3, lr=2.00e-04]
[0] 0:python3*                                                                                  "mc [root@809e0ca1a59e" 19:45 04-Jul-26

Engineers anticipate a steep drop in the perplexity curve within the first 250 iterations of Shard 1 as the ternary weights align with the tokenizer's token distributions.

Intended Use

This foundational model is being developed specifically for robotics applications :

• Real-time control policies
• Multimodal reasoning (text + vision + action tokens)
• Edge deployment with ternary efficiency
• Low-latency physical interaction loops

Monitoring & Updates

Training is actively progressing across all 7 shards . Follow this repository or the linked tokenizer/dataset cards for checkpoint releases, evaluation results, and fine-tuned robotic variants.

Stay tuned — the ternary robotics revolution is just getting started! 🚀