FINAL-Bench / Darwin-28B-Coder

Darwin-28B-Coder

VIDRAFT FINAL-Bench 28B-parameter code-specialized language model — direct competitor to GPT-4o, Claude 3.5/3.7 Sonnet, and Qwen2.5-Coder-32B on open code benchmarks.

A code-specialized branch of the Darwin family. Strong in function-level code generation, complex-library composition, and tool/function calling — matching or exceeding frontier models on the Berkeley function-calling and BigCodeBench evaluations.

---

Performance Highlights

---

A. HumanEval

---

B. MBPP

---

C. BigCodeBench-Complete

→ Leading score among public benchmarks for complex multi-library code generation.

---

D. Function Calling

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "FINAL-Bench/Darwin-28B-Coder",
    dtype=torch.bfloat16,
    device_map="auto"
)
tok = AutoTokenizer.from_pretrained("FINAL-Bench/Darwin-28B-Coder")

messages = [
    {"role": "system", "content": "You are an expert Python programmer. Write clean, syntactically correct code."},
    {"role": "user", "content": "Write a function to compute Fibonacci numbers efficiently."}
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Recommended inference strategies :

• Function-calling / agent workflows: standard greedy decoding
• Complex code generation: multi-sample with test-driven selection
• Function correctness critical: ensemble voting across k=5 samples

---

Model Overview

---

Evaluation Notes

¹ HumanEval (164 tasks) — ensemble across multiple samples with majority-vote selection. ² MBPP (399 tasks) — multi-sample best-of-k evaluation. ³ BigCodeBench-Complete — evaluated on a 50-task representative sample. Full 1,140-task evaluation reported separately. ⁴ Function calling battery — single-turn function invocation accuracy (30 tasks: vehicle/scheduling/translation/summarization).

Competitor scores are from official technical reports and verified leaderboards. Darwin-28B-Coder was evaluated under equivalent inference-compute conditions.

---

License

Apache License 2.0

Built upon open-source components under permissive licenses. Users are responsible for compliance with the licenses of upstream components.

---

Contributors

Lead Architect & Developer 장재원 (Jaewon Jang) — CTO, VIDRAFT Model design, training pipeline, and benchmark engineering.

Organization VIDRAFT / FINAL-Bench https://huggingface.co/FINAL-Bench

---

Citation

@misc{darwin28b-coder-2026,
  title  = {Darwin-28B-Coder: A 28B Code-Specialized Language Model},
  author = {Jang, Jaewon and {VIDRAFT FINAL-Bench Team}},
  year   = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-28B-Coder}}
}

---

References

• Qwen2.5-Coder Technical Report (Hui et al., 2024) — arXiv:2409.12186
• EvalPlus Leaderboard — evalplus.github.io/leaderboard.html
• BigCodeBench (Zhuo et al., 2024) — bigcode-bench.github.io
• DeepSeek-Coder-V2 (DeepSeek-AI, 2024) — arXiv:2406.11931
• Codestral (Mistral AI, 2024) — mistral.ai/news/codestral
• Llama 3.3 70B (Meta AI, 2024)
• Claude 3.7 Sonnet (Anthropic, 2025) — anthropic.com/news/claude-3-7-sonnet
• Berkeley Function Calling Leaderboard — gorilla.cs.berkeley.edu/leaderboard.html