Mungert / DASD-4B-Thinking-GGUF

DASD-4B-Thinking GGUF Models

Model Generation Details

This model was generated using llama.cpp at commit 0c21677e4 .

---

Quantization Beyond the IMatrix

I've been experimenting with a new quantization approach that selectively elevates the precision of key layers beyond what the default IMatrix configuration provides.

In my testing, standard IMatrix quantization underperforms at lower bit depths, especially with Mixture of Experts (MoE) models. To address this, I'm using the --tensor-type option in llama.cpp to manually "bump" important layers to higher precision. You can see the implementation here:
👉 Layer bumping with llama.cpp

While this does increase model file size, it significantly improves precision for a given quantization level.

I'd love your feedback—have you tried this? How does it perform for you?

---

Click here to get info on choosing the right GGUF model format

---

DASD-4B-Thinking

🚀 Introduction

We release DASD-4B-Thinking , a compact yet capable 4B dense language model specialized in long chain-of-thought (Long-CoT) reasoning across mathematics, code generation, and scientific reasoning. DASD-4B-Thinking is post-trained from Qwen3-4B-Instruct-2507 (non-thinking student) and distilled from gpt-oss-120b (teacher) via a distribution-aligned sequence distillation pipeline, achieving strong long-cot reasoning performance with substantially fewer training samples ( 448K ) than many existing larger models.

📊 Performance

---

💡 Why DASD-4B-Thinking Matters

While the community rushes to build distilled reasoning model using massive datasets (often millions of samples), DASD-4B-Thinking proves that distribution alignment matters more than data quantity . It establishes a new baseline for data-efficient distillation , delivering flagship-level reasoning in a 4B model that can run on consumer hardware.

DASD-4B-Thinking democratizes the training recipe:

• Open-Source Model : It achieves State-of-the-Art performance among open-source models of comparable scale and outperforms significantly larger models.

• Extreme Data Efficiency : Achieves these results using only 448K training samples , an order of magnitude fewer than comparable efforts.

• Novel pipeline : It presents a systematic reexamining of sequence-level distillation and introduces a novel distribution-aligned sequence distillation pipeline .

• Open-Source Data : We release the Alibaba-Apsara/Superior-Reasoning-SFT-gpt-oss-120b , allowing the community to reproduce our off-policy temperature-scheduled pipeline:

  • 105K Low-Temperature responses for stability (Stage 1).

  • 330K High-Temperature responses for diversity (Stage 2).

• Proven Scalability : The exact same data recipe generalizes effectively to larger architectures, as demonstrated by our DASD-30B-A3B-Thinking-Preview (MoE), which achieves competitive performance without extra RL.

⚙️ Post-Training Pipeline

DASD-Thinking introduces a new paradigm of Distribution-Aligned Sequence Distillation . This represents an enhanced sequence-level distillation pipeline that incorporates Temperature-scheduled Learning , Divergence-aware Sampling , and Mixed-policy Distillation , achieving efficient capability transfer with a minimal amount of data ( 448K ). Please refer to our report for more details.

⚡ Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Alibaba-Apsara/DASD-4B-Thinking"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

prompt = "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?"
messages = [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=81920,
)

output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
content = tokenizer.decode(output_ids, skip_special_tokens=True)
print(content)

Note: We include the system prompt, as it was used during all training stages. To ensure consistent output quality, we recommend including the same system prompt during actual usage; otherwise, the model's responses may be affected.

For deployment, you can use sglang>=0.4.6.post1 or vllm>=0.8.5 or to create an OpenAI-compatible API endpoint:

• SGLang:

python -m sglang.launch_server --model-path Alibaba-Apsara/DASD-4B-Thinking --context-length 262144

• vLLM:

vllm serve Alibaba-Apsara/DASD-4B-Thinking --max-model-len 262144

💡Best Practices

To achieve optimal performance, we suggest using Temperature=1.0, TopP=1.0 .

📜 Licence

The model weights are licensed under Apache 2.0 License.

⚠️ Limitation

While DASD-4B-Thinking demonstrates remarkable performance across mathematical, scientific, and coding benchmarks, it is currently limited by the absence of tool integration and function calling capabilities. Operating strictly within the text space, the model cannot interact with external interfaces such as code executors or APIs, which constrains its utility in agent-based workflows; however, future iterations aim to bridge this gap by integrating capabilities like knowledge retrieval and tool invocation to support more complex, interactive reasoning tasks.

📚 Citation

DASD-Thinking is developed by Alibaba Cloud, as part of our mission to advance open, efficient, and trustworthy reasoning systems. If you find this work useful in your research or applications, please cite our technical report.

@article{yan2026dasd,
  title={Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning},
  author={Yan, Shaotian and Liu, Kaiyuan and Shen, Chen and Wang, Bing and Fan, Sinan and Zhang, Jun and Wu, Yue and Wang, Zheng and Ye, Jieping},
  year={2026},
  journal={arXiv preprint arXiv:2601.09088},
  url={https://arxiv.org/abs/2601.09088}
} 
    
@article{liu2025where,
  title={Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation},
  author={Liu, Kaiyuan and Yan, Shaotian and Miao, Rui and Wang, Bing and Shen, Chen and Zhang, Jun and Ye, Jieping},
  journal={arXiv preprint arXiv:2512.20908},
  year={2025}
}

We welcome collaboration, feedback, and community contributions to push the boundaries of what small models can reason about—transparently and responsibly.

---

🚀 If you find these models useful

Help me test my AI-Powered Quantum Network Monitor Assistant with quantum-ready security checks :

👉 Quantum Network Monitor

The full Open Source Code for the Quantum Network Monitor Service available at my github repos ( repos with NetworkMonitor in the name) : Source Code Quantum Network Monitor . You will also find the code I use to quantize the models if you want to do it yourself GGUFModelBuilder

💬 How to test :
Choose an AI assistant type :

• TurboLLM (GPT-4.1-mini)
• HugLLM (Hugginface Open-source models)
• TestLLM (Experimental CPU-only)

What I’m Testing

I’m pushing the limits of small open-source models for AI network monitoring , specifically:

• Function calling against live network services
• How small can a model go while still handling:
  • Automated Nmap security scans
  • Quantum-readiness checks
  • Network Monitoring tasks

🟡 TestLLM – Current experimental model (llama.cpp on 2 CPU threads on huggingface docker space):

• ✅ Zero-configuration setup
• ⏳ 30s load time (slow inference but no API costs ) . No token limited as the cost is low.
• 🔧 Help wanted! If you’re into edge-device AI , let’s collaborate!

Other Assistants

🟢 TurboLLM – Uses gpt-4.1-mini :

• **It performs very well but unfortunatly OpenAI charges per token. For this reason tokens usage is limited.
• Create custom cmd processors to run .net code on Quantum Network Monitor Agents
• Real-time network diagnostics and monitoring
• Security Audits
• Penetration testing (Nmap/Metasploit)

🔵 HugLLM – Latest Open-source models:

• 🌐 Runs on Hugging Face Inference API. Performs pretty well using the lastest models hosted on Novita.

💡 Example commands you could test :

1. "Give me info on my websites SSL certificate"
2. "Check if my server is using quantum safe encyption for communication"
3. "Run a comprehensive security audit on my server"
4. '"Create a cmd processor to .. (what ever you want)" Note you need to install a Quantum Network Monitor Agent to run the .net code on. This is a very flexible and powerful feature. Use with caution!

Final Word

I fund the servers used to create these model files, run the Quantum Network Monitor service, and pay for inference from Novita and OpenAI—all out of my own pocket. All the code behind the model creation and the Quantum Network Monitor project is open source . Feel free to use whatever you find helpful.

If you appreciate the work, please consider buying me a coffee ☕. Your support helps cover service costs and allows me to raise token limits for everyone.

I'm also open to job opportunities or sponsorship.

Thank you! 😊