NVIDIA Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model Family at 3B and 8B
NVIDIA released Nemotron-Labs-Sigma, a family of 14 open checkpoints at 3B and 8B parameters spanning base, instruct, few-step distilled, and self-speculative variants. The model family performs continuous diffusion in a learned 16-D embedding space, with a 4,096-token context that extends to 8,192 for Sigma-8B-instruct. NVIDIA reports Sigma-8B base averaging 71.16 on its 6-task suite versus 70.22 for NLD-8B, while the instruct 10-task average is 59.14, slightly below NLD-8B's 59.92. The self-speculative 8B checkpoint runs 3.46x faster than autoregressive decoding on one GB200 with SGLang, according to the company.
Why it matters: It gives readers a concrete open-weight model release with architecture details, benchmark results, deployment constraints, and inference-speed claims for continuous diffusion language models.