[D] Position paper: using hallucination as a construction instrument to distill task-specific cognitive kernels from frontier models [D]

Background: I am a software developer, not an ML researcher. This started from a practical question — why do AI coding tools send proprietary client code to remote servers when the task only requires Swift? Following that question produced this framework.

The core proposal

Current approaches to LLM distillation ask: how do we preserve as much general capability as possible in a smaller model?

This paper asks the opposite: can we deliberately eliminate all capability except one task — and use the point where everything outside that task becomes incoherent as the measurable boundary of a deployable kernel?

The instrument for finding that boundary is hallucination. Specifically: the field uses entropy-guided methods to detect where a model's knowledge boundary is. This paper proposes running the same signal in reverse — as a construction instrument during distillation rather than a detection tool after training.

For coding tasks the measurement is objective: compilation rate and pass@k. You distill until Python pass@k stays high and COBOL compilation rate hits zero. That gradient is the boundary. The compiler is the arbiter — not a subjective assessment.

What existing research supports this

• Task-specific capabilities concentrate in sparse attention head sets. Zeroing out five math-specific heads degrades math performance by up to 65% while leaving other tasks largely unaffected. This suggests boundary discovery via targeted distillation is more tractable than naive weight entanglement analysis implies. (Bair et al. 2026, arxiv.org/abs/2603.03335)

• Knowledge boundary discovery via entropy-guided RL already exists. This paper proposes running it in the opposite direction — moving the boundary inward deliberately rather than detecting where it already is. (Wang & Lu 2026, arxiv.org/abs/2603.21022)

• Machine unlearning (forget loss + retain loss) provides the negative reinforcement mechanism for capability retirement — driving deprecated patterns below operational utility without weight deletion.

• A 770M parameter model distilled from a 540B teacher outperformed the teacher on specific tasks using 80% of training data — distillation consistently beats training from scratch for task-specific performance.

What is not validated

• Whether the two-curve gradient is clean enough to be practically useful or whether within-domain weight entanglement makes it too noisy
• Whether the measurement methodology generalises cleanly beyond code to domains without formal correctness criteria
• The precise protocol parameters

This is a research agenda not a result. The paper is explicit about what is validated and what is hypothesis. It includes an appendix with self-critique and responses to the likely technical objections including the weight entanglement challenge.

The framework also proposes a complete lifecycle mechanism — upskilling kernels when technology evolves and downskilling deprecated capabilities through negative reinforcement — and a bidirectional boundary mapping approach that would produce a complete skill inventory of a frontier model.

Paper: https://osf.io/9u5bc/overview?view_only=15f6aaedb7a6499bbfdb610113ef07b6