TOPSD: A New Method to Prevent Teacher Drift in Multilingual Reasoning Distillation
Researchers propose Teachable On-Policy Self-Distillation (TOPSD) to address Teacher Drift in multilingual reasoning distillation, where the teacher model follows student errors instead of correcting them. TOPSD identifi...