Pluto-AI-Labs / Atlas-Coder-0.5B-GGUF

---

About This Repository

This repository contains GGUF quantizations of Atlas-Coder-0.5B — a coding-specialized language model fine-tuned from Qwen2.5-Coder-0.5B base using QLoRA on 80K decontaminated code instructions.

These GGUF files are optimized for CPU inference on consumer hardware using llama.cpp , Ollama , and LM Studio . No GPU required.

For full training details, benchmark results, and the original FP16 model, see the Atlas-Coder-0.5B main model page .

---

Available Quantizations

Which Quantization Should I Use?

• Q8_0 → Maximum fidelity, RAM not a concern, you want the closest to FP16 output
• Q6_K → Best all-rounder. This is the one to use if you are unsure
• Q5_K_M → Excellent quality at a smaller footprint. Barely distinguishable from Q6_K for most tasks
• Q4_K_M → Minimum RAM usage. Still surprisingly capable for a 400MB file

All four files run comfortably on any modern laptop with 4GB+ RAM.

---

Quick Start

Ollama

# Download the model file first, then create a Modelfile
cat > Modelfile << EOF
FROM ./Atlas-Coder-0.5B-Q5_K_M.gguf

SYSTEM "You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning."

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER num_predict 512
EOF

ollama create atlas-coder -f Modelfile
ollama run atlas-coder "Write a Python function to reverse a linked list"

LM Studio

1. Open LM Studio
2. Go to Search tab
3. Search for Siddh07ETH/Atlas-Coder-0.5B-GGUF
4. Download your preferred quantization
5. Load and chat

llama.cpp (CLI)

# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Run inference
./llama-cli \
  -m Atlas-Coder-0.5B-Q5_K_M.gguf \
  -p "<|im_start|>system\nYou are Atlas-Coder, an elite AI coding assistant.<|im_end|>\n<|im_start|>user\nWrite a Python binary search function.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 \
  --temp 0.3 \
  --top-p 0.9 \
  --repeat-penalty 1.1 \
  --no-display-prompt

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="Atlas-Coder-0.5B-Q5_K_M.gguf",
    n_ctx=1024,
    n_threads=4,
    verbose=False,
)

prompt = """<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code.<|im_end|>
<|im_start|>user
Write a Python function to check if a number is prime.<|im_end|>
<|im_start|>assistant
"""

output = llm(
    prompt,
    max_tokens=512,
    temperature=0.3,
    top_p=0.9,
    repeat_penalty=1.1,
    stop=["<|im_end|>", "<|im_start|>"],
)

print(output["choices"][0]["text"])

---

Recommended Generation Settings

---

Chat Template

Atlas-Coder uses ChatML format. Always wrap prompts in this structure:

<|im_start|>system
You are Atlas-Coder, an elite AI coding assistant created by Siddharth N.R. You write clean, efficient, and well-documented Python code. You specialize in code generation, completion, debugging, refactoring, algorithm implementation, and software engineering reasoning.<|im_end|>
<|im_start|>user
YOUR QUESTION HERE<|im_end|>
<|im_start|>assistant

---

Original Model

This GGUF repository is derived from the full FP16 model:

➡️ Siddh07ETH/Atlas-Coder-0.5B

The original model page contains:

• Full training methodology and pipeline details
• Dataset mixture and decontamination details
• Benchmark results (EvalPlus / HumanEval+ — publishing soon)
• FP16 and 4-bit inference code examples
• Research context and engineering decisions

---

About Atlas-Coder-0.5B

Atlas-Coder-0.5B is a coding-specialized LLM instruction-tuned from Qwen2.5-Coder-0.5B base (not instruct) using QLoRA on a Tesla T4. Key training highlights:

• 80K samples across 4 code datasets, including 50K execution-verified OSS-Instruct samples
• n-gram Jaccard decontamination against HumanEval — benchmark scores are honest
• Response-only loss masking — gradients only on assistant code output
• Trained entirely on free Kaggle T4 GPU — reproducible on consumer hardware
• 3-layer checkpoint recovery — survived Kaggle's 12-hour session limit

Part of the Pluto AI research project by Siddharth N.R.

---

Performance

Benchmarks (EvalPlus / HumanEval+) are currently being run on the original FP16 model and will be published on the main model page shortly.

GGUF quantization quality vs FP16 baseline (approximate):

---

Author

Siddharth N.R. Final-year B.Tech — AI & Data Science Pluto AI Research

---

Related Models

---

Citation

@misc{atlascoder2026,
  author    = {Siddharth N.R.},
  title     = {Atlas-Coder-0.5B: A QLoRA-Trained Sub-1B Coding Model from Base},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Siddh07ETH/Atlas-Coder-0.5B}
}

---

License

Apache 2.0 — see LICENSE . Base model Qwen2.5-Coder-0.5B is also Apache 2.0.