Live intelligence

Latest AI updates

4 updates

Research

MonoLaDN: Language-Guided DeNoising for Monocular 3D Object Detection

MonoLaDN is a training framework that transfers geometric reasoning from a frozen vision-language model into a monocular 3D detector. It encodes the VLM's structured reasoning about occlusion, depth, dimension, and angle into queries, perturbs them with noise, and trains the detector to reconstruct the clean reasoning. The deployed detector remains purely vision-based, adding no inference latency or parameters. Experiments on KITTI 3D and nuScenes report consistent improvements over state-of-the-art detectors and good generalization to new domains.

arxiv22h agoScore 78

monocular 3D object detectionvision-language modelsknowledge distillation

Research Editor selected

Shuffled OPSD: When Semantic Consistency Hurts On-Policy Self-Distillation?

This paper introduces Shuffled OPSD, a variant of on-policy self-distillation that shuffles the words of the shared problem statement while keeping the teacher-only worked example intact. Across three math reasoning benchmarks and three Qwen3 model scales, it improves mean accuracy over both intact-target and Standard OPSD, with the largest gain of 3.52 points on Qwen3-1.7B. Controls show that shuffling the teacher-only example hurts accuracy, while word-level shuffling of the problem statement helps, challenging the assumption that semantic coherence is always beneficial.

Why it matters: Provides concrete, recent experimental evidence that disrupting input structure can improve self-distillation, offering a novel insight for AI training methodology.

arxiv1d agoScore 85

on-policy self-distillationknowledge distillationmathematical reasoning

Tutorial Editor selected

Build efficient agents with knowledge distillation on Amazon SageMaker AI

This post demonstrates how to use sequence-level knowledge distillation (SeqKD) to fine-tune a smaller student model (Qwen3.5-4B) to replace a frontier teacher (GLM-5.3) for a shopping agent, reducing cost and latency. It covers generating high-quality trajectories via rejection sampling, encoding them for supervised fine-tuning, running serverless training on SageMaker AI, and evaluating the student with a low-cost pre-check before full end-to-end evaluation. The approach is practical for API-only teachers and includes code snippets for the entire pipeline.

Why it matters: Provides a concrete, actionable guide for a cost-saving AI technique (SeqKD) with a real-world agentic use case, including implementation details on AWS.

AWS Machine Learning Blog3d agoScore 92

knowledge distillationagentic AImodel efficiency

Research Editor selected

Turing MoE Study Reports Strong Specialist Integration and 2.86× Faster Consolidation

A new paper combines calibrated multi-teacher on-policy distillation with execution optimizations for dynamically routed mixture-of-experts models. On Turing-20B-A2B, the authors report retaining at least 90.16% of measured specialist gains in every evaluated domain and outperforming shared supervised fine-tuning and Qwen3-8B on four reasoning-enabled mathematics and code benchmarks at a matched response cap. A PPU-based profile shows consolidation step time falling from 398.7 to 139.3 seconds, with the results attributed to the complete training and execution framework rather than a single component.

Why it matters: The study offers concrete evaluation and execution results for researchers seeking to combine specialist models while reducing MoE post-training costs.

arxiv6d agoScore 86

Turing-20B-A2BMixture-of-expertsKnowledge distillation