reaperdoesntknow / Dualmind-Qwen-1.7B-Thinking

Dualmind-Qwen-1.7B-Thinking

Claude Opus 4.6 Reasoning Traces → 1.7B via DualMind SFT

Convergent Intelligence LLC: Research Division

What This Is

A 1.7B model trained on 2.5M+ tokens of Claude Opus 4.6 reasoning traces using the DualMind SFT methodology. The training data comes from Opus-4.6-Reasoning-3000x-filtered — a curated dataset of extended reasoning chains from Anthropic's most capable model, with refusals removed.

This is the Opus variant of the DualMind family. Where the base DualMind model was trained on LogicInference data, this model absorbs the reasoning patterns of Claude Opus 4.6 — longer chains, more nuanced self-correction, and richer deliberative structure. The Opus teacher produces qualitatively different reasoning than synthetic logic datasets: it backtracks, hedges, reconsiders, and synthesizes in ways that reflect genuine uncertainty navigation rather than pattern completion.

The base model is Disctil-Qwen3-1.7B — already DISC-refined and sitting in the middle of the DistilQwen distillation chain — giving it a strong structural foundation before the Opus reasoning signal is applied.

Architecture

Training

Training Dynamics

The loss curve shows clean convergence across 7.4 epochs with no signs of overfitting — eval loss (1.406) remains below final training loss (1.455). The 6.8 percentage point gain in token accuracy reflects genuine absorption of the Opus reasoning structure, not memorization.

Why Opus Traces

The Opus-4.6-Reasoning dataset captures something that synthetic datasets don't: the way a frontier model navigates genuine uncertainty. Opus doesn't just solve problems — it reasons about its own confidence, backtracks when a line of thought weakens, and synthesizes across multiple attempted approaches. When you distill from these traces, the student doesn't just learn to produce correct answers. It learns the shape of deliberation .

This is the DualMind thesis in practice: the cognitive loop (explore → examine → respond) isn't an architectural trick. It's a training signal. When the teacher naturally exhibits multi-phase reasoning, the student absorbs that structure through standard SFT.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "reaperdoesntknow/Dualmind-Qwen-1.7B-Thinking"
)

messages = [
    {"role": "user", "content": "What happens to information that falls into a black hole? Walk me through the paradox."}
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=2048,
    do_sample=True,
    top_p=0.9,
    temperature=0.7,
    repetition_penalty=1.15
)

print(tokenizer.decode(output[0], skip_special_tokens=True))

Generation Tips

• Temperature 0.6–0.8 — the Opus reasoning traces have natural variance in them. Don't flatten it with low temperature.
• Repetition penalty 1.1–1.2 — prevents looping during extended reasoning chains.
• Max tokens 1024–2048 — trained at 4096 max seq, so it can go long. The Opus signal rewards longer generation windows.
• The model may produce multi-phase reasoning naturally (exploring, then reconsidering, then concluding). This is the intended behavior — the DualMind cognitive loop emerging from the training signal.

Model Lineage

Qwen3-1.7B (base)
  → DiStil-Qwen3-1.7B-uncensored (uncensored SFT)
    → Disctil-Qwen3-1.7B (DISC refinement)
      → Dualmind-Qwen-1.7B-Thinking ← you are here
           ↑
    Opus 4.6 reasoning traces (2.5M tokens, DualMind SFT)

DualMind Family Comparison

Same methodology, different teachers, different capabilities. The LogicInference variant is more mechanical. The Opus variant is more deliberative. TopologicalQwen is the full TKD pipeline with BV decomposition. They're complementary — different facets of the same cognitive architecture.

DualMind Collection

Full collection: DualMind on HuggingFace

Papers

• Structure Over Scale: Proof-Weighted Knowledge Distillation — DOI: 10.57967/hf/8165. The DistilQwen methodology paper.
• Three Teachers to Dual Cognition — DOI: 10.57967/hf/8184. The DualMind extension: ghost imprinting and multi-teacher convergence.

License

Apache 2.0

Mathematical Foundations: Discrepancy Calculus (DISC)

This model's training pipeline is grounded in Discrepancy Calculus — a measure-theoretic framework that treats singularities as primary structure rather than pathology. Full theory: "On the Formal Analysis of Discrepancy Calculus" (Colca, 2026; Convergent Intelligence LLC: Research Division).

The Core Operator:

$Df\left(x\right)=\underset{\epsilon \downarrow 0}{\lim }\frac{1}{\epsilon }{\int }_x^{x+\epsilon }\frac{\mid f\left(t\right)-f\left(x\right)\mid }{\mid t-x\mid }dtDf(x)\; =\; \backslash lim\_\{\backslash varepsilon\; \backslash downarrow\; 0\}\; \backslash frac\{1\}\{\backslash varepsilon\}\; \backslash int\_x^\{x+\backslash varepsilon\}\; \backslash frac\{|f(t)\; -\; f(x)|\}\{|t\; -\; x|\}\backslash ,\; dt$ D f ( x ) = ε ↓ 0 lim ​ ε 1 ​ ∫ x x + ε ​ ∣ t − x ∣ ∣ f ( t ) − f ( x ) ∣ ​ d t

For smooth $f$: $Df(x) = |f'(x)|$. For rough $f$: $D$ localizes irregularity to null sets while preserving integral structure.

The Mesh Fundamental Identity — every BV function decomposes as:

$f\left(b\right)-f\left(a\right)=\underset{\text{smooth (AC)}}{\underset{⏟}{{\int }_a^b{f}^{\prime }\left(x\right)dx}}+\underset{\text{jumps}}{\underset{⏟}{\underset{x\in J_f}{\sum }\Delta f\left(x\right)}}+\underset{\text{Cantor drift}}{\underset{⏟}{D^{c}f\left(I\right)}}f(b)\; -\; f(a)\; =\; \backslash underbrace\{\backslash int\_a^b\; f\text{'}(x)\backslash ,dx\}\_\{\backslash text\{smooth\; (AC)\}\}\; +\; \backslash underbrace\{\backslash sum\_\{x\; \backslash in\; J\_f\}\; \backslash Delta\; f(x)\}\_\{\backslash text\{jumps\}\}\; +\; \backslash underbrace\{D^c\; f(I)\}\_\{\backslash text\{Cantor\; drift\}\}$ f ( b ) − f ( a ) = smooth (AC) ∫ a b ​ f ′ ( x ) d x ​ ​ + jumps x ∈ J f ​ ∑ ​ Δ f ( x ) ​ ​ + Cantor drift D c f ( I ) ​ ​

Standard knowledge distillation captures only term 1. Topological Knowledge Distillation (TKD) preserves all three by treating the teacher's output distribution as a BV function and computing discrepancy energy, jump sets, and gap energy density before training begins.

Citation

@misc{colca2026dualmind,
  title={Three Teachers to Dual Cognition: From Knowledge Distillation to Emergent Reasoning},
  author={Colca, Roy},
  year={2026},
  doi={10.57967/hf/8184},
  publisher={Convergent Intelligence LLC: Research Division}
}

Convergent Intelligence LLC: Research Division — 49 models, 22,598+ downloads across the portfolio. Full portfolio | DualMind Collection | DistilQwen Collection