muradil211 / AetherSearch_DPO

🔭 AetherSearch DPO

A preference-aligned search agent for retrieval-augmented reasoning

Continued from AetherSearch SFT with direct preference optimization on the AetherSearch_DPO dataset.

🏠 Project · 🧠 SFT checkpoint

📚 Dataset: AetherSearch_DPO

🔌 Bring your own retriever. AetherSearch DPO is a search-agent policy, not a self-contained question-answering service. The host runtime must execute each <search>...</search> request and return evidence inside <information>...</information> .

✨ Highlights

• 🎯 Preference aligned : trained with DPO to favor stronger search and answer trajectories.
• 🔎 Search native : produces explicit reasoning, retrieval requests, and grounded answers.
• 🔁 Multi-step retrieval : can request additional evidence when one search is insufficient.
• ⚡ Compact backbone : retains the accessible 3B-scale Qwen2 architecture.

🧠 How it works

Question
   |
   v
<think>reason about what is missing</think>
   |
   v
<search>focused retrieval query</search> -----> Search / RAG backend
   ^                                                  |
   +---- <information>retrieved evidence</information>+
   |
   +-- repeat the search loop when more evidence is needed
   v
<answer>evidence-grounded final answer</answer>

The model produces the reasoning, search, and answer spans. The surrounding runtime parses each completed <search> span, runs retrieval, appends the result as <information> , and resumes generation until the model emits an <answer> span.

📊 Model at a glance

🚀 Quick start

Load the checkpoint with:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "muradil211/AetherSearch_DPO"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.config.use_cache = True

For end-to-end use, wrap generation in the retrieval loop shown above and preserve the XML protocol exactly.

⚠️ Limitations

Generated searches and answers can be incorrect, unsupported, or unsafe. Retrieval quality, evidence validation, and answer verification remain the caller's responsibility.

📜 Terms

No additional blanket license is asserted here. Review the Qwen2.5-3B-Instruct license , the AetherSearch SFT terms, and the AetherSearch_DPO dataset terms before redistribution or downstream use.