Explore tools, prompts, skills, news and resources related to GDN.
Prompt works related to GDN
Latest articles, prompts, use cases and editorial content about GDN
Alibaba has released Qwen4-Max, a proprietary sparse Mixture-of-Experts model with a 2.4T-parameter backbone, 71B active parameters per token, and a 1M-token context window. It accepts text, images, and video and is avai...
Zyphra has released ZONOS3-Mini, an Apache 2.0 text-to-speech model with 800M total parameters and about 100M active parameters in a sparse Mixture-of-Experts design. It supports zero-shot voice cloning, word-level deliv...
A new arXiv study finds that storing cached recurrent state in bf16 with round-to-nearest makes hybrid reasoning models more verbose and increases 32K-token cap hits. Across six open hybrid models and 590 competition-mat...
Thinking Machines released Inkling 2 Small, a 594B-total/17B-active multimodal MoE, trained from scratch in 68 hours on 8.4T tokens (NVFP4). It scores 31 on the Artificial Analysis Intelligence Index, beating many same-s...
This paper introduces a hierarchy of word problems for state tracking, from simple immutable assignments to general finite-state tracking, and proves that transformers cannot length-generalize on all aperiodic problems w...
SpecOPD introduces a branch-parallel framework for on-policy distillation in speculative decoding, enabling draft models to be trained without sequential draft-and-verify rollouts. By packing independent branches into pr...
A new paper demonstrates that large language models can directly read and edit the weights of smaller neural networks, enabling tasks like bug fixing and weight generation without traditional training pipelines. The appr...
DAMI is a new initialization method for gated linear attention models that configures gate parameters and input activations before training to improve long-context retrieval. It assigns geometric timescales across heads...
SparseGDN introduces a hardware-aware framework for Gated Delta Networks that reduces computation and memory traffic by combining structured intra-chunk approximation with input-dependent recurrent-event selection. Evalu...
The paper introduces Agentic Soft-Token Compression (ASTC), a training-free method that uses soft tokens within an LLM agent to compress accumulated context for long-horizon tasks. ASTC leverages the model's native...
Qwen Team has officially open-sourced FlashQLA, a high-performance operator library specifically designed for the Gated Delta Network (GDN), the linear attention layer that powers the entire Qwen series, includ...