Research
TOPSD: A New Method to Prevent Teacher Drift in Multilingual Reasoning Distillation
Researchers propose Teachable On-Policy Self-Distillation (TOPSD) to address Teacher Drift in multilingual reasoning distillation, where the teacher model follows student errors instead of correcting them. TOPSD identifies where drift begins using teacher-student likelihood and only applies supervision to prefixes that pass verification. Experiments across ten languages and three model scales show consistent improvements over conventional OPSD, especially in low- and medium-resource languages.
self-distillationmultilingual reasoningteacher-student learning