DualMind
Collection
One model, two voices. Explore reasons freely. Examine critiques adversarially. Response synthesizes. • 5 items • Updated
Convergent Intelligence PRO
reaperdoesntknow
AI & ML interests
About Us
Mission
Convergent Intelligence advances original research in discrepancy calculus, adaptive systems, and applied AI, translating those insights into client controls, playbooks, and leadership-ready briefs.
Recent Activity
updated a collection about 5 hours ago
DualMind updated a collection about 5 hours ago
DualMind updated a model about 6 hours ago
reaperdoesntknow/DualMinded-Qwen3-1.7B-GGUFOrganizations
updated a
collection about 5 hours ago
updated a
model about 6 hours ago
published a
model about 6 hours ago
updated a
model about 6 hours ago
published a
model about 6 hours ago
updated a
collection about 8 hours ago
updated a
model about 8 hours ago
published a
model about 8 hours ago
posted an update about 8 hours ago
Post
37
# Three Teachers, One Student: Dual-Cognition Reasoning at 1.7B
We distilled Qwen3-30B-A3B into 1.7B students that critique their own reasoning. H100, BF16, Apache 2.0. Here's the pipeline.
**Stage 1 — Three Teachers, Three Profiles.** Same 30B base, three variants: Instruct (structured output), Thinking (extended deliberation), Coder (STEM decomposition). Each distillation uses proof-weighted KD — 2.25× amplified loss on reasoning tokens, decaying to 1.1×. The student learns *where to think harder*, not just what to output.
**Stage 2 — Topology-Aware KD (TKD).** Standard KD treats the teacher's distribution as smooth. Language isn't smooth — it has topic shifts, reasoning pivots, register changes. We use Discrepancy Calculus to detect these structural boundaries, then amplify loss at jumps (3σ threshold) and cut training windows at low-discrepancy positions. The student preserves the teacher's structural knowledge, not just surface statistics.
**Stage 3 — Ghost Imprinting.** Sequential distillation from different teachers leaves residual fields in weight space that neither teacher put there individually. The Cantor component of BV decomposition, applied to parameters. Models distilled Thinking→Coder exhibit deliberation patterns from the Thinking teacher that survived Coder overwriting. Emergent capability from structural residuals.
**Stage 4 — DualMind.** One model, two voices, shared weights:
The multi-model collision array collapsed into a single architecture. Role tokens, no extra parameters.
For the full method:
reaperdoesntknow/DualMind_Methodolgy
doi:10.57967/hf/8184.
We distilled Qwen3-30B-A3B into 1.7B students that critique their own reasoning. H100, BF16, Apache 2.0. Here's the pipeline.
**Stage 1 — Three Teachers, Three Profiles.** Same 30B base, three variants: Instruct (structured output), Thinking (extended deliberation), Coder (STEM decomposition). Each distillation uses proof-weighted KD — 2.25× amplified loss on reasoning tokens, decaying to 1.1×. The student learns *where to think harder*, not just what to output.
**Stage 2 — Topology-Aware KD (TKD).** Standard KD treats the teacher's distribution as smooth. Language isn't smooth — it has topic shifts, reasoning pivots, register changes. We use Discrepancy Calculus to detect these structural boundaries, then amplify loss at jumps (3σ threshold) and cut training windows at low-discrepancy positions. The student preserves the teacher's structural knowledge, not just surface statistics.
**Stage 3 — Ghost Imprinting.** Sequential distillation from different teachers leaves residual fields in weight space that neither teacher put there individually. The Cantor component of BV decomposition, applied to parameters. Models distilled Thinking→Coder exhibit deliberation patterns from the Thinking teacher that survived Coder overwriting. Emergent capability from structural residuals.
**Stage 4 — DualMind.** One model, two voices, shared weights:
<explore> — free derivation, speculation
<examine> — adversarial self-critique
<response> — clean synthesisThe multi-model collision array collapsed into a single architecture. Role tokens, no extra parameters.
For the full method:
reaperdoesntknow/DualMind_Methodolgy
doi:10.57967/hf/8184.
updated a
collection about 8 hours ago
updated a
model about 8 hours ago
published a
model about 8 hours ago
updated a
collection about 9 hours ago
updated a
model about 9 hours ago
published a
model about 9 hours ago
updated a
collection about 9 hours ago