artindnr / tea-4bit

🍵 Tea — 4-bit

This repo contains a 4-bit precision version of artindnr/tea , a full fine-tune of microsoft/phi-4 for question answering and long, multi-turn assistant conversations, with fine-tuning focused on Farsi (Persian) conversational ability.

Unlike GGUF quants, this is a safetensors checkpoint quantized to 4-bit precision with Unsloth (bitsandbytes nf4 backend), meant to be loaded directly with 🤗 Transformers or Unsloth — not with llama.cpp.

Looking for GGUF quants for llama.cpp / Ollama / LM Studio instead? See artindnr/tea-gguf (F16, Q8_0, Q5_K_M, Q4_K_M).

Model Details

• Base model: artindnr/tea (full fine-tune of microsoft/phi-4 , 14B parameters)
• Quantized by: artindnr , using Unsloth
• Format: safetensors (4-bit, bitsandbytes nf4)
• License: MIT
• Languages: Farsi (primary conversational focus), English, and general multilingual support

How to Use

Generation with 🤗 Transformers

The checkpoint is already stored in 4-bit, so it loads directly — no BitsAndBytesConfig needed on your end, device_map="auto" handles placement.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "artindnr/tea-4bit"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="auto",
)

USER_PROMPT = "تو کی هستی و اسمت چیه؟"

messages = [
    {"role": "user", "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Generation with Unsloth (faster inference/fine-tuning)

pip install unsloth

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="artindnr/tea-4bit",
    max_seq_length=8192,
    load_in_4bit=True,
    dtype=None,  # auto-detect
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "user", "content": "تو کی هستی و اسمت چیه؟"},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Unsloth also lets you use this checkpoint as a starting point for further QLoRA fine-tuning if you want to adapt tea further while keeping it in 4-bit.

Intended Use

Same as artindnr/tea : Farsi-first conversational assistance, question answering, and long multi-turn assistant deployments — this repo specifically targets lower-VRAM GPU inference and fine-tuning via Transformers/Unsloth, as an alternative to the GGUF quants for CPU/llama.cpp-based deployment.

Limitations

• 4-bit quantization trades off some accuracy for memory footprint; expect some quality degradation versus the full-precision artindnr/tea , particularly on nuanced or long-context Farsi generation.
• This checkpoint requires a CUDA GPU and bitsandbytes — it is not intended for CPU inference or llama.cpp; use artindnr/tea-gguf for that.
• Inherits all limitations of the base artindnr/tea model and the underlying microsoft/phi-4 checkpoint, including possible hallucinated facts.
• No formal safety fine-tuning beyond what is inherited from the base model has been applied; use appropriate safeguards in production settings.

License

This model is released under the MIT License , consistent with artindnr/tea and the base microsoft/phi-4 model.

Citation

If you use tea in your work, please cite:

@misc{tea,
  title  = {tea: A Farsi-Focused, Full Fine-tune of Phi-4 for QA and Long-form Assistance},
  author = {artindnr},
  year   = {2026},
  url    = {https://huggingface.co/artindnr/tea}
}

Acknowledgements

Built on top of artindnr/tea , itself a full fine-tune of microsoft/phi-4 . 4-bit quantization via Unsloth .