eulogik / Bharat-Tiny-LLM

Bharat-Tiny-LLM 🇮🇳

Hinglish AI — trained on a Mac Mini M4, runs on ₹8,000 phones.

---

The Story

I trained an AI to speak Hinglish on a Mac Mini. No cloud compute. No GPU cluster. Just $750 of Apple Silicon and a lot of chai.

This is Bharat-Tiny-LLM — a Qwen2.5-1.5B model fine-tuned on 376K Hinglish conversations. Quantized to 828 MB (Q4), it delivers ~57 tokens/second on a Mac Mini M4 and is small enough to run on an ₹8,000 smartphone.

Most AI models today are skyscrapers — built by billion-dollar companies, requiring data centers, and serving only English. We built a bicycle. A tiny, efficient, open-source model that speaks the way 600 million Indians actually talk: Hinglish.

What Makes This Different

---

Model Details

Architecture

• Base Model: Qwen2.5-1.5B (Apache 2.0)
• Fine-Tuning Method: LoRA (16 layers, rank 8, alpha 16)
• Training Hardware: Mac Mini M4, 16GB unified memory
• Training Duration: ~3.3 days for 76K iterations
• Quantization: 4-bit (Q4) via MLX — 4.501 bits/weight
• Total Trainable Parameters: 5.276M (0.342% of base)

Base model details (click to expand)

This model is fine-tuned from Qwen2.5-1.5B (Apache 2.0), one of the few small models with native multilingual support (29 languages including Hindi). Unlike English-only models (like SmolLM2, which we tried first — val loss plateaued at 1.32), Qwen2.5 already understands Hindi morphology. LoRA fine-tuning on Hinglish data was enough to make it conversational.

Performance

---

Quick Start

Using mlx-lm (macOS / Apple Silicon)

pip install mlx-lm

python3 -c "
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load('eulogik/Bharat-Tiny-LLM')
sampler = make_sampler(temp=0.7)

prompt = '<|im_start|>user\nChai peete hain?<|im_end|>\n<|im_start|>assistant\n'
response = generate(model, tokenizer, prompt=prompt, max_tokens=200, sampler=sampler)
print(response)
"

Platform Support

---

Example Outputs

---

Training Data

We curated 376,040 conversations from 5 open-source datasets, deduplicated and formatted as chat messages:

All data was:

• Trimmed to 512 characters per message (to fit within model context)
• Converted to messages format with user / assistant roles
• Split 95% train / 5% validation

---

Training Details

Hyperparameters

Base model:    Qwen2.5-1.5B (bf16 from mlx-community)
Method:        LoRA (16 layers, rank 8, alpha 16)
Batch size:    4
Seq length:    512
Learning rate: 5e-5 (with cosine schedule)
Iterations:    76,420
Optimizer:     AdamW
Warmup:        100 steps
Seed:          42

Training Curve

Val Loss
1.25 |  █
1.20 |  ██
1.15 |  ███
1.10 |  ████
1.05 |  █████
1.00 |  ██████
0.95 |  ████████
0.90 |  ██████████
0.85 |  ████████████
0.80 |  ██████████████
0.78 |  ████████████████  ← Best at 75K iters
     +------------------------
     0   20K   40K   60K   80K

Speed Optimization

---

Limitations

1. Not fully converged — Only 76K iterations; val loss was still decreasing at 0.781. More training (target: 110K) would help.
2. Occasional English drift — Some responses switch entirely to English. Needs more Hinglish-only examples.
3. Telugu leakage — IndicVault Hindi data contains some Telugu examples that leak into responses.
4. Emotional/career topics — Responses to queries about jobs, relationships, or mental health can be confused or repetitive.
5. Short context — 512 token limit prevents multi-turn conversations or long-form generation.
6. MLX format only — The Q4 quantized model is in Apple MLX format. We plan to release a transformers-compatible version.
7. Not instruction-tuned — This is a base model fine-tuned on conversations, not RLHF/DPO-tuned. Responses are helpful but not aligned.

---

Roadmap

• Phase 0: Fine-tune Qwen2.5-1.5B with LoRA on Hinglish data
• Phase 1: Quantize to Q4 (828 MB) for edge deployment
• Phase 2: Extend to 8 Indian languages (Tamil, Telugu, Bengali, Marathi, etc.)
• Phase 3: DPO/RLHF for quality improvement
• Phase 4: BharatTiny-Bench evaluation suite
• Phase 5: Web demo with Gradio
• Phase 6: Android / Raspberry Pi 5 deployment

---

Why "Bharat-Tiny-LLM"?

600 million Indians speak Hinglish — a natural mix of Hindi and English. But every LLM today is either English-only or pure Hindi. There is no model that understands "Yaar, kal ka match dekh liya? Rohit ne toh maara!"

We built this because:

• No one builds for edge devices in India. Everyone targets data centers.
• No one speaks Hinglish. Everyone speaks either English or "pure" Hindi.
• No one does it on a budget. Everyone burns millions in cloud compute.

We're building the bicycle while everyone else builds skyscrapers.

---

Citation

@misc{bharat-tiny-llm-2026,
  title = {Bharat-Tiny-LLM: Hinglish Edge AI Trained on a Mac Mini},
  author = {Gautam Kishore},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/eulogik/Bharat-Tiny-LLM}
}

---

License

Apache 2.0 — Free for commercial and research use. The base model (Qwen2.5-1.5B) is also Apache 2.0.

---

Built with chai ☕, patience 🧘, and a Mac Mini M4 💻 — somewhere in India.